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Foreword 



“The central fact is that we are planning agents.” 

(M. Bratman, Intentions, Plans, and Practical Reasoning, 1987, p. 2) 



Recent arguments to the contrary notwithstanding, it seems to be the case 
that people — the best exemplars of general intelligence that we have to date — 
do a lot of planning. It is therefore not surprising that modeling the planning 
process has always been a central part of the Artificial Intelligence enterprise. 
Reasonable behavior in complex environments requires the ability to consider 
what actions one should take, in order to achieve (some of) what one wants — 
and that, in a nutshell, is what AI planning systems attempt to do. 

Indeed, the basic description of a plan generation algorithm has remained 
constant for nearly three decades: given a desciption of an initial state /, a 
goal state G, and a set of action types, find a sequence S of instantiated 
actions such that when S is executed in state /, G is guaranteed as a result. 
Working out the details of this class of algorithms, and making the elabora- 
tions necessary for them to be effective in real environments, have proven to 
be bigger tasks than one might have imagined. 

Initially, plan formation was approached as a formal process, in particular, 
in Green’s work on planning as theorem proving [59]. But beginning in the 
early 1970s, the focus of the planning community shifted to system develop- 
ment, starting with the STRIPS planning system, which was used, amongst 
other things, to enable Shakey- the- Robot to form plans to push blocks around 
the halls of SRI International. STRIPS was followed by a series of ever larger, 
more complex, and, alas, often more ad hoc planning systems. A major break 
occurred in the late 1980s, marked by the publication of three key papers: 
Chapman’s paper on the TWEAK formalism [28], Pednault’s paper on the 
ADL formalism [104], and McAllester and Rosenblitt’s paper on the SNLP 
algorithm [94]. These papers were intended not to add functionality to known 
planning methods, but rather to capture the essential elements of these known 
methods in a readily analyzable fashion. As such, they signaled the begin- 
ning of an effort, still ongoing within the planning community, to address the 
planning problem more systematically, giving greater care to analyzing the 
relationships among alternative representations and algorithms. 
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Foreword 



With this shift, the field of planning was ready for a comprehensive book 
on planning that gives the “lay of the land” , showing what methods exist, how 
they are related, and what their relative strengths and weaknesses are. In- 
telligent Planning: A Decomposition and Abstraction Based Approach is that 
book. General AI textbooks can typically devote only a chapter or two to the 
topic of planning, and so can give only a suggestion of the range of issues that 
arise in plan generation, and of the range of solutions that have been pro- 
posed in response. In contrast, this book explores the issues and the solutions 
in depth, giving a careful and thorough analysis of each. It takes the perspec- 
tive that effective planning relies on two techniques that are fundamental in 
computer science — decomposition (also often known as divide-and-conquer) 
and hierarchical abstraction — and it uses this perspective to structure the 
material very effectively. 

Comprehensive monographs are already available for several other subar- 
eas of AI, such as natural-language processing [4] and machine learning [86]. 
These books have played an important role, bringing together the major ideas 
of their respective areas to provide a solid platform on which further research 
can be based. Intelligent Planning: A Decomposition and Abstraction Based 
Approach will do the same for AI planning. As a textbook, it will doubtless be 
a valuable resource for graduate students and their professors. It will also be a 
valuable resource for researchers actively working in the field of AI planning, 
and those in other areas who need to know about AI planning, as it provides 
ready access to the basic computational tools — representations, algorithms, 
and analyses — on which further research into the nature of planning will rely. 



Martha Pollack 




Preface 



Those who triumph 
Plan at their headquarters 
Considering many conditions 
Prior to a challenge. 

Those who are defeated 
Plan at their headquarters 
Considering few conditions 
Prior to a challenge. 

Much planning brings triumph 
Little planning brings defeat 
How much more so 
With no planning at all! 

Observing a planning process 
I can see triumph and defeat. 

Sun Tzu [400-320 B.C., China], The Art of Strategy^, Chapter One 

Planning has captivated human interest for generations. The above quote, 
translated from a classical Chinese text on the art of strategy, underscores 
this fascination. To live our lives, we have to deal with a huge number of 
problems, many of which require careful planning. However, until recently 
the problem of how to plan has not been a subject of systematic study. 
This situation, however, has completely changed with the dramatic progress 
of computer technology and the enormous success of Artificial Intelligence 
(AI). _ 

This book is a monograph on Artificial Intelligence Planning (AI Plan- 
ning), an active research and applications field for the past several decades. 
As a research field, planning can be defined broadly as the study of actions 
and changes, covering topics concerning action and plan representation, plan 
synthesis and reasoning, analysis of planning algorithms, plan execution and 
monitoring, plan reuse and learning. Lately there has been a dramatic in- 
crease of interest in automatic and semi-automatic planning in AI and other 
related fields. It has been demonstrated that research in planning is of great 
importance to most subfields of AI as well as general computer science, en- 
gineering, management science, and cognitive science. 



^ The quotes in this book are adapted from an excellent translation by R.L. Wing 
[137]. The book is also known as The Art of War. 
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Features 

To cover the entire field in a single volume is impossible. In writing the book, 
I have chosen to focus on a clear, thorough coverage of key areas of classical 
AI planning. Classical AI planning is concerned mainly with the generation 
of plans to achieve a set of pre-defined goals in situations where most relevant 
conditions in the outside world are known, and where the plan’s success is 
not affected by changes in the outside world. As we will see, the planning 
task is extremely hard even under these situations. 

The book’s main purpose is to build more intelligence on a set of ba- 
sic methods for reasoning about and generating plans. This is done by first 
explaining these basic methods, then developing advanced techniques to en- 
hance them using problem decomposition and abstraction. In addition, the 
book presents techniques for analyzing and comparing planning algorithms. 

In order to be accessible to readers from a wide variety of backgrounds, 
the book takes a ground-zero approach. It begins with a gentle introduction 
and is self-contained; most key algorithms and techniques can be found in the 
book itself, rather than referenced from other sources. As a result, it requires 
minimal preconditions on the part of the reader and will benefit not only 
seasoned researchers, but undergraduate and graduate students, as well as 
researchers in other related fields such as mechanical engineering, business 
administration and software project management. Many useful algorithms 
and techniques are compiled in a single volume, expressed in a common syn- 
tactic framework. Many illustrations, examples, algorithms, analyses, tables, 
and references help make the explanation clear. Each chapter ends with a 
background survey of the current state of research, the source of the material 
under discussion, and an exploration of open problems. 



Contents and Intended Audience 

The book consists of three main parts. Part I, “Representation, Basic Algo- 
rithms and Analytical Techniques,” lays the foundation. This part provides 
a general introduction to plan representation, generation, and analysis. It 
reviews past and current representations and algorithms in AI planning and 
general computer science that are basic but foundational. 

Parts II and III of the book present my own contribution to planning in 
the past decade. Both parts develop advanced planning techniques that are 
based on the basic planning algorithms and methods from Part I. Part II, 
“Problem Decomposition and Solution Combination,” presents a complete 
suite of analysis and algorithm tools for decomposition planning. Decom- 
position planning refers to the task of breaking apart a complex problem 
into smaller pieces and solving them individually, and later combining their 
solutions into a global solution. 
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IX 



Part III is entitled “Hierarchical Abstraction,” and presents a theory of 
plan generation using the idea of abstraction. Abstraction refers to the task 
of solving a problem by tackling the most important components first, then 
using the “skeleton” solution as a guide to solve the rest of the problem. 
This part presents analysis, comparisons between planning with and without 
abstraction, methods for generating an abstraction hierarchy automatically, 
properties of hierarchical task network (HTN) planning, and effect abstrac- 
tion. 

This book can be used as a one-semester or one-quarter textbook in an 
Introduction to AI course, or an AI Planning course. It can also be used as 
a reference book for graduate seminar courses. As a teaching resource, it can 
be used in both graduate or undergraduate courses. As a reference resource, 
it can be used by researchers and practitioners in AI, computer science, en- 
gineering, and other related fields. Knowledge of basic data structures, logic, 
and algorithm analysis would be helpful, but is not strictly required. Any 
programming experience would also be very useful. 
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1. Introduction 



1.1 The Problem 

Planning is about producing changes through actions. The planning tasks 
we face are, of course, diverse. What routes to follow to deliver an express 
package from one city to another? What actions to take in order to maximize 
one’s investment rewards while reducing the amount of risk? What sequence 
of commands to give to a computer-controlled machine tool for manufacturing 
an automobile part? What update operations to follow for retrieving certain 
information from a large, distributed database? The list goes on. 

With the advance of computer technology many planning activities can 
now be automated. We can envision a computer-aided planning toolkit for 
generating parts of a plan automatically, for selecting actions among a large 
set of alternatives, for looking for bugs in a complex plan, and for suggesting 
how to reduce the cost of a given plan. 

The central aim of this book is to design a library of automatic plan- 
ning methods using techniques in Artificial Intelligence. These methods are 
collections of planning algorithms resulting from extensive research in this 
area by the author and his co-workers, and are built on experiences that 
other Artificial Intelligence researchers have accumulated in the past several 
decades. All methods are aimed at making planning more effective. That is, 
the plan generation and reasoning process are to become more efficient, and 
the quality of the plans is to become higher (lower costs and risks, and higher 
reliability). Although these methods do not form a complete list of all algo- 
rithms needed for constructing an automatic planner, they nevertheless cover 
many of the important aspects of automatic planning. In the next section, 
we discuss these aspects at a fairly general level. 



1.2 Key Issues 

What problem-solving abilities must a planner possess in order to successfully 
compose a workable plan? We begin by considering the following scenario of 
a planning task: a robot is given the task of painting both a ceiling and a 
ladder. The robot formulates the following plan: it first fetches a can of paint 
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Fig. 1.1. A painting example 



and two brushes. It then places the ladder under the ceiling and climbs up 
to the top. After painting the ceiling with one brush, it switches to another 
and proceeds to paint the ladder (see Figure 1.1). In this problem, we want 
the planner to generate a good plan, and to do so as quickly as possible. 

The first issue we consider is efficiency. An intelligent planner must be 
able to compose a plan efficiently. The issue of efficiency is of major concern 
here because of the potentially large number of action combinations as plans; 
given ten actions that agents are able to perform, a naive planner might 
have to look through 10^° sequences of actions before finding a correct ten- 
step plan! An equally important issue is the quality of a plan generated, 
especially when the planner must choose among many alternative plans in 
order to find one with low cost. Here again the number of alternative plans 
might be prohibitive for a naive method to be applied. A planning tool that 
is able to compose a good plan quickly might be called an effective planner. 

One way to construct a plan is incrementally; that is, add one plan step 
at a time until all final goals are achieved. Constraints can be posted in the 
process to ensure that the plan is free from harmful interactions among its 
steps. Finally, the plan steps could be added in either a forward or a backward 
manner. In the former case a plan grows from the current situation to the 
final situation, and in the latter the direction is reversed. These incremental 
methods to planning, on which virtually all AI planning research builds, 
might be called basic planning algorithms. 

The central thesis of this book is that to be effective, a planning algorithm 
must go beyond a basic one. It should exploit the structures of the problem at 
hand and work along the lines of well-established human intuitions. One such 
intuition is divide- and- conquer^ whereby a complex problem is divided into 
several more-or-less independent parts and those parts are solved separately. 
Another intuition is hierarchical abstraction, with which a distinction is made 
between parts of a problem that are difficult to solve and those that are rel- 
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atively easy to solve. These parts are organized into the levels of a hierarchy, 
which can be used by a planning algorithm to make the planning process 
more effective. In this book we present formalized computational methods 
for realizing both of these intuitions. 

Divide- and- Conquer. By the intuition of divide- and-conquer, an intelligent 
planner ought to have the ability to decompose a complex problem domain 
into subcomponents. Problem decomposition will make it possible to attack 
the individual problem components separately^ thereby improving planning 
efficiency. In the painting example, the planner might decide that painting 
the ceiling and painting the ladder can be planned for separately. To do 
so incurs a number of intriguing issues. First of all, decomposition involves 
recognizing the parts of the problem domain that are independent to some 
degree. This process is nontrivial due to the potentially large number of 
possible decompositions; for a problem domain with ten possible descriptive 
features, there are potentially 2^^, or over one thousand possible ways just 
to divide these features into two groups! The second issue involves how the 
solutions to the individual components can be successfully combined. This 
again is a complex problem. 

During the combination process, the need arises to reason about conflicts 
among multiple plans. For the painting robot, if the robot can only hold 
one brush at a time, then a resource conflict occurs between the part of 
the plan for painting the ceiling and the part of the plan for painting the 
ladder. Similarly, if the wet paint from painting the ladder precludes one 
from climbing up, then another conflict occurs because performing the former 
negates a precondition of the latter, which requires that the ladder be dry. 
With the number of conflicts mounting in a large plan, efficient methods for 
conflict resolution are necessary. 

Another issue involved with the solution-combination process is that dif- 
ferent solutions might contain redundant parts when combined. Some of these 
redundancies can be removed to further improve the overall quality of the 
plan. In the painting domain, if the two goals, painting the ceiling and paint- 
ing the ladder, are planned for separately, the two plans, when combined, 
might contain two identical steps for fetching the ladder. These two steps can 
be merged into one. This leads to a strategy called plan merging, whereby 
parts of a plan can be merged together to yield a plan with lower cost. 

Yet another issue in problem-decomposition is that a subproblem might 
have several alternative solutions. When the solutions to the subproblems are 
considered together again, the choice of one solution to a subproblem might 
affect how the solutions to other subproblems are selected. Some collective 
choices may lead to a globally incorrect plan; others may yield a dramatically 
more costly plan. This plan selection problem therefore requires an intelligent 
solution that is more than a random selection from each solution set. 

Hierarchical Abstraction. As pervasive as divide- and-conquer, hierarchical 
abstraction is another often-used human intuition in attacking planning prob- 
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lems. This method essentially uses the distinctions between important aspects 
of a planning problem from those that are mere details. By so doing a plan- 
ner could first work with a much smaller version of the problem and use the 
resultant solution to constrain subsequent problem solving. For our painting 
planner, an abstract plan might involve three actions at a high level of descrip- 
tion in a sequence: get the paint and brushes; paint the ceiling; finally, paint 
the ladder. When all aspects of the problem are taken into account, these 
steps serve as a framework around which the entire plan could be fleshed 
out. 

Improvements to the Basic Planners. Each of the above techniques, problem- 
decomposition and hierarchical abstraction, requires that a basic planner be 
available. In problem-decomposition, a basic planner is needed for solving 
each decomposed subproblem. In abstraction, a basic planner is needed for 
fleshing out a more abstract plan. It is therefore also an important task to 
improve the efficiency of the basic planners. 

For a basic planner that works backwards from the goal, its efficiency is 
greatly affected by the number of actions that can achieve each given subgoal. 
The larger the number of such actions the more costly is the planning process. 
One way to reduce the number is to classify certain effects of an action as 
primary effects. These are the effects that determine the main purpose of 
an action. Others are considered as secondary. For example, “having the 
brush” is a primary effect of fetching the brush, and “the agent’s hand is 
non-empty” might be considered secondary. By focusing on primary effects, 
a planner might lose its ability to generate plans for all problems, but we 
might still be happy as long as it could generate good plans for most of the 
problems more quickly. 

Another technique we use to improve the efficiency of a basic planner is to 
consider available resources collectively. In the painting example, the planner 
might be facing a large number of available brushes for selection. If we do 
not have adequate knowledge about them initially, and if few of the brushes 
are useful in the end, then a naive commitment to a particular brush might 
result in a bad plan, a fact which may not be discovered until much later. 
An intelligent planner might decide to delay the choice on a particular brush, 
and choose one only when there is enough information for such a selection. 
This may allow a failed search path to be discovered early, and also allows 
many similar plans to be considered collectively. 

In this book, we present these techniques in two separate chapters: Chap- 
ter 13 in Part I, and Chapter 5 in Part III. 



1.3 Planning Versus Scheduling 

A field closely related to planning is known as scheduling. Indeed, authors 
in Operations Research often use the two terms interchangeably. However, 
there is an important difference. 
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Fig. 1.2. A PERT network for a painting project. Each arc is an activity, and each 
node is a state (or event). The heavy lines illustrate a critical path 



Scheduling grew out of efforts to formalize optimizations of large combi- 
natorial problems. A characteristic scheduling problem is to assign machines 
to machined parts (e.g., automobile parts) given that each machined part 
could be handled by more than one machine. The complication arises from a 
number of factors: the difference in costs associated with each machine and 
machined part, various constraints on delivery deadlines and the availability 
of machines and storage resources. Often the aim of scheduling is to optimize 
certain value functions subject to a set of constraints. 

One way to visualize the above optimization problem is to consider the 
computation being done on a network of tasks. Each task can be considered 
a high-level description of an action to be performed, yet without time and 
machine resources assigned to it. Although different assignment of resources 
can yield a solution with a different degree of satisfaction in the optimality 
criteria and constraints, the inter-relations between the tasks are more or 
less determined already. In other words, this network of tasks, often called 
a PERT (Project Evaluation and Review Technique) network, is given as 
input. 

A PERT network is a set of tasks or activities constrained by a partial 
order. A partial order specifies which tasks must occur before which others. 
It is acyclic — if A is before B then B cannot be before A (see Figure 1.2). 
Many types of analysis can be performed on such a network; a typical one is 
CPM (Critical Path Method). In the figure, a critical path (boldfaced path) 
is determined by a longest path from the start state to the finish state. This 
path information is useful in determining the shortest time required to finish 
the project, should no unexpected events occur. 

As emphasized above, finding a critical path in a PERT network assumes 
that a plan structure (a PERT network) is already given. Subsequent com- 
putation is performed to optimize a plan based on its structure. A natural 
question to ask, then, is how to construct the network of activities — a gen- 
eralized plan — in the first place. This is the planning problem, a problem 
that has been more or less ignored by the field of Operations Research, but 
is nevertheless equally important and fascinating! 
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Obviously even for a plan generation algorithm there must be some form 
of initial input. We must know something about the general capabilities of 
the plan-execution agent, and about the outside world. This is exactly the 
approach taken by many AI planning researchers. As we will see later, this 
knowledge amounts to a specification of action representations and world- 
state specifications. When a plan is assembled, these actions might interact 
in different ways, calling for sophisticated methods for reasoning about the 
effects of actions. In contrast, in a scheduling problem, the action interac- 
tions are assumed known and sorted out; the only remaining task is to assign 
resources to each action to optimize a cost function. Reasoning about ac- 
tions and their effects is one of the main distinctions between planning and 
scheduling. 

It should also be noted that it may be very beneficial to interleave planning 
and scheduling in an intelligent way. A plan can be constructed partially when 
an assignment of resources on the planning tasks is attempted. This approach 
has the obvious advantage that if a plan cannot lead to a good schedule, it is 
abandoned early. It is nevertheless much clearer conceptually to draw a line 
between planning and scheduling; planning precedes scheduling. This is the 
approach taken in this book. 



1.4 Contributions and Organization 

This book introduces advanced computational methods for dealing with plan- 
ning problems. We will strive for our methods to be as general as possible so 
that they can be applied under different underlying representational schemes. 
However, the real focus of our presentation will be on the algorithmic aspects 
of planning; we present intelligent algorithms for the above planning tasks 
and support our claim for their effectiveness theoretically and empirically. 

In particular, the book is divided into the following three parts, illustrated 
in Figure 1.3. 

Part 1: Basic Representations and Algorithms. In this part we present pre- 
liminary representations and basic algorithms for planning (Chapter 2). 
We also discuss some useful analytical methods for comparing planning 
methods (Chapter 3) and motivate our subsequent presentations. In ad- 
dition, we give a systematic summary of useful supporting algorithms 
for plan reasoning (Chapter 4), so that the contents of the book are self- 
contained. Finally, we conduct a case study of an application of constraint 
satisfaction to planning, showing how natural it is to extend some ba- 
sic planning algorithms to include more elaborate functionalities (Chap- 
ter 5). 

Part 2: Planning Problem Decomposition. This part deals with the issue of 
problem decomposition and solution combination. The important ques- 
tions are: 
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Fig. 1.3. An outline of the issues addressed in this book 



— how to decompose a problem domain into separate parts, so that a 
plan can be made in each part concurrently (Chapter 6); 

— how to resolve conflicts that arise among the different solution plans 
that are to be combined (Chapter 7); 

— how to recognize opportunities for optimizing plans by merging repet- 
itive actions (Chapter 8); 

— how to select plans from a pool of potential candidates? The selection 
process may be guided by the need to resolve conflicts (Section 9.1), 
or by the need to optimize plans (Section 9.2). 

Part 3; Planning with Hierarchical Abstraction. This part deals with a for- 
malization of hierarchical planning and planning with abstraction. The 

important questions are: 

— how to generate and refine plans in any given abstraction hierarchy 
(Chapter 10); 

— how to automatically generate abstraction hierarchies by separating 
the important and unimportant aspects of a problem (Chapter 11); 

— how to plan using a predefined library of task reduction schemata, and 
what properties of the library will ensure effective planning (Chap- 
ter 12); 
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— how to trade-off completeness (i.e., ability to find solutions for all solv- 
able planning problems) with efficiency, by abstracting the effects of 
operators (Chapter 13). 



1.5 Background 

The problems of planning have fascinated generations of AI researchers. A 
main theme has been to build systems that could generate plans from scratch. 
Early work include Newell and Simon’s GPS [101], Green’s QA3 system 
for theorem proving [59], Pikes, Nilsson and Hart’s Strips [46], Sussman’s 
Hacker [130], and Warren’s Warplan [143]. Around the 1970s, more so- 
phisticated planning systems started to appear beyond research labs, includ- 
ing Sacerdoti’s Noah [114] Tate’s Nonlin [132], Vere’s Deviser [140], and 
Wilkins’ SiPE [147]. Applications of these systems ranged from office-building 
construction to space-craft activity planning. 

The understanding of planning algorithms and analytical techniques be- 
gan to mature around the mid-1980s. This movement has largely been driven 
by the need to understand these systems better, by dividing a complex plan- 
ning system into manageable components, and then studying each component 
thoroughly. In other fields of computer science, one could, for example, spend 
an hour of classroom time in teaching sorting algorithms (see, for example, 
[2]) and then send the students off for implementation. But one could not do 
that with planning. The systems mentioned above required a certain degree 
of simplification and formal characterization first. 

With this motivation, in his AI Journal paper in 1987, Planning for Con- 
junctive Goals [28], Chapman presented proofs that planning in general is 
undecidable, and that to decide whether a plan is correct is NP-hard (likely 
to require exponential time in the worst case). Pednault [103], working in- 
dependently, provided a logically precise account of plan correctness under a 
general representation scheme, and discussed various ways of implementing 
search. Other treatments of this topic, to name a few, included work by Lan- 
sky [88], Drummond and Currie [39], Tenenberg [134], Minton, Bresina and 
Drummond [95], Gupta and Nau [60], McAllester and Rosenblitt [94], By- 
lander [24], Backstrom [14], Kambhampati [70], Knoblock [80], Barrett and 
Weld [17], and Yang [150] 

Today, several excellent surveys on planning are available. These include 
Wilkins’ description of the SiPE system [147], Weld’s article in AI Maga- 
zine [144], Allen, Hendler and Tate’s collection Readings in Planning [5], 
book chapters in AI textbooks by Russell and Norvig [112], Dean, Allen, and 
Aloimonos [135], Winston [148], Rich and Knight [110], Tanimoto [131], and 
Charniak and McDermott [29]. 

^ This list is by no means exhaustive; in subsequent chapters we will survey other 
relevant work in more detail. 
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By choosing to focus on algorithms and analysis techniques for plan rea- 
soning, we could not manage to cover many other important topics in the 
scope of this book. These other topics include reactive planning, plan exe- 
cution and monitoring [115, 1, 23], case-based planning [82, 163], planning 
in natural language understanding [145, 4], reasoning about actions and be- 
liefs in logic [90, 61, 109], multi-agent planning [40], plan reuse and machine 
learning in planning [97, 73], GraphPlan[ 21], and decision theory and con- 
trol [35]. Each of these topics deserves thorough and complete coverage. 




Part I 



Representation, Basic Algorithms, 
and Analytical Techniques 




Overview 



The elements of strategy: 

First, measurements 
Second, estimates 
Third, analysis 
Fourth, balancing 
Fifth, triumph 

The situation gives rise to measurements 
Measurements give rise to estimates 
Estimates give rise to analysis 
Analysis gives rise to balancing 
Balance give rise to triumph 

(Sun Tzu, The Art of Strategy^ Chapter Four) 



This part contains four chapters, serving to set the stage for the rest of the 
book. In Chapter 2 we first outline a simple language used to describe a plan- 
ning domain, then describe some basic algorithms for constructing a plan. In 
Chapter 3 we show how to analyze and compare planning algorithms based on 
a few parameters. Throughout the book we use some traditional algorithms 
used in computer science, and for the book to be self-contained, these tech- 
niques are briefly reviewed in Chapter 4. Finally, in Chapter 5, we conduct a 
case study of the application of one of the supporting techniques, constraint 
satisfaction in planning. We show how to combine constraint satisfaction 
techniques with a basic planner, and illustrate the potential effectiveness of 
the resulting system. 




2. Representation and Basic Algorithms 



2.1 Basic Representation 

One of the early representational methods for planning domains is known 
as the Strips representation [102]. This representation models actions as 
operations on a database, which records the current state of affairs in the 
world. Its simplicity is one of the major reasons for its vast popularity in 
many theoretical and practical work in planning. It has also been shown that 
many of the more exotic representational schemes, such as those involving a 
limited form of first-order logic representations, can be easily and naturally 
derived from the Strips representation. In this book, we will employ the 
Strips representation for the exposition of our major ideas and algorithms, 
and discuss possible extensions to these algorithms where the representational 
languages become more sophisticated. 

2.1.1 Domain Description 

A Strips domain description consists of a subset of first-order predicate 
language L for describing the domain, and an operator set O for describing 
the abilities of the agent. T is a restricted language consisting of predicate 
symbols pi, negation -i, constant symbols c^, and variable symbols Xi. The 
terms of L are the constants and variables in L. An atom is an expression of 
the form 

p{t\ ^ , tfi) 1 

where P is an n-ary predicate and the parameters ti are terms. The ground 
atoms are the atoms where all terms are constants. The literals (also called 
propositions) include all atoms and their negations. Further, for any literal 
p, -I -ip is equivalent to p. 

As an example, consider a description of a painting domain described in 
Chapter 1. Here the predicate symbols are 

Painted, Havebrush, Havepaint, Haveladder, Dry, Handempty, . . . 

The constant symbols are 

Door, Brush, Ladder, Ceiling, paint, 
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And the variables are ?brush, ?paint, . . . 

If one assigns truth values (TRUE or FALSE) to every literal in a domain 
description such that, whenever p is assigned TRUE, ^p is assigned FALSE, 
then one obtains a state. Because the number of literals needed for defining a 
single state is often too large to handle, we often talk about state descriptions. 
A state description is a subset S of all literals in a domain such that every 
literal in S is given a truth value “TRUE,” their negations are false, and the 
rest of the literals in the domain definition are assumed unknown. In effect, 
S describes a set of states. With this machinery we can describe an initial 
state as a set of literals representing their conjunction. For example, in the 
painting domain an initial state description might be 

{-•Painted(Door), -iPainted(Ladder), Dry(Ladder), Handempty} 

For succinctness, we often use the term initial state for an initial state de- 
scription. 

In Strips representation a goal description is also represented as a con- 
junction of literals. For example, in the painting domain, if we wish to have 
both the door and the ladder painted, we might say 

{Painted(Door), Painted(Ladder)} 

where the comma represents logical and. 

The second element, O, in a Strips representation is a set of operators. 
Each operator a in O consists of four elements: 

Operator Name. This is a list of symbols, OpName(tx, ^ 2 ? • • -)i where the first 
symbol is the name of the operator, and the rest are variables and con- 
stants appearing in descriptions of preconditions and effects of the oper- 
ator. 

Preconditions of a. This is a set of literals representing their conjunction, 
with the intention that these literals must be true immediately before the 
operator is applied. The preconditions of a are represented as Pre(a). 
Effects of a. This is a set of literals representing their conjunction, with the 
intention that these literals will hold after the operator is applied. The 
effects of a are represented as Eff(o;). 

Operator Cost. This is a real value denoting the cost of performing the action 
denoted by the operator. 

As an example in the painting domain, several operators are shown in Ta- 
ble 2.1. 

If an operator description contains variables, it is called partially instan- 
tiated. If all variables are replaced by constants, then it is called a ground 
instance of the operator, and the instance of the operator is said to be fully 
instantiated. 

Let 5 be a set of fully instantiated literals; S describes a set of states where 
all literals in S hold. Let s be a fully instantiated operator, s is applicable 
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Table 2.1. Operator definition for the painting example 



Precondition 


Effect 


Cost 


get-brush 


Handempty, 


Havebrush(?6), 


1.0 


Dry(?6) 


-iHandempty 




paint-ceiling 


Havebrush(?6), 


-iDry(Ceihng), 


2.0 


Dry (Ladder), 


-Dry(?6), 




Havepaint (Paint) 


Painted (Ceiling) 




paint-ladder 


Havebrush(?6), 


-■Dry (Ladder), 


3.0 


Havepaint(Paint) 


-Dry(?6), 

Painted(Ladder) 




return-brush 


Havebrush(?6) 


-iHavebrush(?6) , 
Handempty 


1.0 



to 5 if all preconditions of s hold in 5. The application of s to 5 results in 
another set T of literals describing the successor states of S reachable by s. 
T is defined as follows. Let Del be the set of literals I in S such that is a 
member of Eff(s) (recall that = 1). That is, Del is the set of literals of S 
deleted by s. Then T is defined as: 

T:={S- Del) U EflF(s) 

T is called a successor state description of S after s. 

A Strips planning problem is defined as a triple {init, goal^ O), where init 
is a set of initial state literals, goal are the goal literals, and O is the set of 
planning operators. A sequence of fully instantiated operators II is called a 
solution to a Strips problem, where each operator is also known as a step. 
In this sequence, the first step is applicable to inil the step is applicable 
to the successor state description obtained after the (i — 1)^^ step, and every 
literal in goal holds in the successor state description after the last step. 77 is 
also called a total-order plan due to its sequential nature. A total-order plan 
satisfying the above definition is said to be correct 

2.1.2 Partial-Order and Partial-Instantiation Plans 

Not every precedence ordering between plan steps in a total-order plan is 
necessary for maintaining its correctness. Sometimes a set of total-order plans 
can be compressed into a structure known as a partial-order plan. A partial- 
order plan consists of a partially ordered set of steps, along with a set of 
constraints on these steps. A step is a version of a planning operator in 
which some variables are replaced by constants. The steps are transformed 
fi'om planning operators and inserted into the plan by a plan generation 
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Fig. 2.1. A partial-order plan 



algorithm. The ordering constraints state various precedence relation among 
the steps, and the variable-binding constraints state whether two variables, 
or a variable and a constant, can designate the same constant. 

We describe each of these components in turn. 

Step Orders. Mathematically, a partial order R on a set is a binary relation 
such that 

— R is irreflexive. That is, for every element a in f/, (a, a) is not in R. 

— R is transitive. That is, for all elements a, 6, and c in U, if (a, b) and (6, c) 
are both in R, then (a, c) is also in R. 

Graphically, a partial order RonU can be denoted as a network, where every 
element of C/ is a node and every pair in R is an edge. The definition of a 
partial order implies that the network does not contain any cycle. 

In a partially ordered plan, the precedence relation on plan steps is a 
partial-order relation. As an example. Figure 2.1 is a partial-order plan for 
painting the ceiling. We use si^sj to denote that for steps and sj, (si, sj) 
is a member of the partial-order relation. If (sj,Si) is not a member of the 
partial-order relation, then we say that it is consistent for to be before Sj 
in il, and denote it by Consistently (sii—>Sj). 

For simplicity, we also assume that in every plan there are two special 
steps Sinit and Sfinish^ where the former represents the initial state, and the 
latter the goal state. Sinu is ordered before all other operators and has an 
empty set of preconditions. Its effects are the literals in the initial state. In 
a similar way, s finish is ordered after all steps in the plan and has an empty 
effect set. Its preconditions are the goal literals. 

A partial-order plan represents a set of total-order plans, where each total- 
order plan is a linear sequence of steps in the partial-order plan such that the 
partial-order relation in the latter is not violated by the sequence. Figure 2.2 
shows two total-order plans corresponding to the partial-order plan shown in 
Figure 2.1. 

Variable- Binding Constraints. In addition to the ordering constraints be- 
tween steps, a partial-order plan also contains a set of variable-binding con- 
straints of the form lx ^ly^ where ?x is a variable in some step in the plan 
and ly is either a variable or a constant in the step. For example, in a paint- 
ing plan a variable-binding constraint might be lb ^ Brushi where Brushi is 
a particular brush which might be unusable by the painting agent. 
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Notice order , 




Fig. 2.2. Two total orders of a partial order plan 



Another type of variable- binding constraint is of the form lx =?y. These 
constraints force the instantiation of lx to be the same as that for ?y, and 
are known as codesignation constraints. 

Causal Links. Finally, a partial-order plan might contain a set of causal 
links — relations between steps that serve as a record for precondition estab- 
lishment. A causal link (s^ Sj) states that step asserts a precondition 
literal p for Sj. That is, 

— p is a precondition of Sj , 

— p is an effect of s^, and 

— Si^Sj holds in the partial order plan. 

In a ceiling-painting plan, a causal link might be 

(get-brush paint -ceiling), 
where p = Havebrush (Brush). 

Putting it together, a partially ordered plan U consists of a set of steps 
Steps(/7), partial ordering constraints Order (iJ), variable- binding constraints 
Binding(iJ) and a set of causal links C-Links(il). If Order(iJ) defines a 
partial-order relation on plan steps, then we say that the ordering rela- 
tion in the plan is consistent. Likewise, if the variable- binding constraints 
Binding (il) do not imply that, for some variable ?x, lx then we say 
that these constraints are consistent. 



2.2 Basic Planning Algorithms 

We will spend the rest of the chapter discussing plan generation algorithms. 
There are many ways to classify a planning algorithm, of which we will discuss 
four representative ways. These four classifications are made based on both 
the representation of a plan and the direction in which a partially completed 
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plan grows. The first dimension of the classification distinguishes between 
total-order and partial-order plans. The second dimension distinguishes the 
direction of operator chaining; it is forward if new operators are added at the 
end of a plan, and backward if they are added at the beginning of a plan. 

These two dimensions create four possible combinations. Depending on 
the choice, one can have a total-order, backward-chaining algorithm or a 
partial-order, backward-chaining algorithm. Likewise, one can have a a total- 
order, forward- chaining algorithm or a partial-order, forward-chaining algo- 
rithm. Each algorithm can have a different performance behavior in any 
given domain. Below, we only describe two of the four combinations, namely, 
partial-order, backward-chaining algorithms and total-order, forward-chain- 
ing algorithms. The other two combinations can be similarly constructed. 



2.3 Partial-Order, Backward-Chaining Algorithms 

One way to generate a plan for solving a Strips planning problem is to 
construct a partial-order plan by incrementally adding all plan components. 
In a basic implementation of a partial-order, backward- chaining planner, the 
preconditions of plan- steps are achieved one at a time. In each iteration, a 

Table 2.2. The POPLAN algorithm 



Algorithm Poplan(0, 

Input: A set of planning operators O, and an initial plan Hjnit consisting of a 
start step and a finish step and a constraint that the start step be before the finish 
step; 

Output: A correct plan if a solution can be found. 

1 OpenList := { 

2 repeat 

3 77:= lowest cost plan in OpenList; 

4 remove 77 from OpenList; 

5 if Correct(77)=TRUE then return(77); 

6 else 

7 if Threats-Exist(77) then 

8 Let t be a threat; Succ := Resolve-Threat (7, 77); 

9 else 

10 Succ ;= Establish-Precondition(77); 

1 1 end if 

12 Add all successors in Succ, generated in Steps 8 or 10, to OpenList; 

13 end if 

14 until OpenList is empty; 

15 return(Fail); 
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General Direction of Plan Growth: ^ 



Start — >■ Finish 
\ 



Subgoal G = Precondition of Finish 



Add operator B 



Subgoal P= precondition of A 




Fig. 2.3. Backward-chaining search in a space of plans 



Start — ^ B — >• Finish 



Subgoal Q = precondition of B 



\ 

Add operator A 


Start — ^ A — Finish 



precondition of a step is selected to be achieved, and operators and plan steps 
that can achieve the precondition are identified. The precondition is then 
achieved by inserting a new causal link in the plan. Subsequently, threats to 
this causal link are removed. This process repeats until every precondition of 
every step in the plan has an associated causal link, and all negative threats 
(see below for a definition) in the plan are removed. 

Table 2.2 shows a top-level version of a partial-order, backward-chaining 
algorithm. The initial plan consists of the start-step/finish-step pair. In steps 
5, 7, 8 and 10, the algorithm uses a number of subroutines which we will 
flesh out in order of appearance. Figure 2.3 demonstrates the plan- generation 
process. 



2.3.1 Correctness 

The correctness of a partial-order plan can be defined in terms of that of 
a total-order plan. Recall that a partial-order plan iJ corresponds to a set 
of totally ordered and fully instantiated plans. Let this set be Instances(iJ). 
We say that a partially ordered plan II is correct if the set Instances (iJ) is 
nonempty, and if every member of Instances (iJ) is correct. 

The above definition can not be used directly to implement the subroutine 
Correct (), since in general the size of Instances(il) might be very large; 
specifically, the number of instances is n! if the partial-order plan has n 
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unordered steps. Thus, we resort to some other means of indirectly checking 
the correctness of a plan. 

Open Preconditions. 

One way to implement the subroutine Correct () is to rely on the information 
provided by causal links. First, we say that a precondition p of a step s in 
a plan U is an open precondition if there does not exist a causal link 
{si s) in the plan. 

Threats. 

We now define a threat to a causal link as a step that can nullify the link. 
Threats come in two types, negative and positive. To define a threat, we first 
need to introduce the notion of unification of two literals. 

Literals l\ and I2 are unifiable if they can be made identical by replacing 
some variables by either some other variables or constants. For example, 
P(?x, ly) and P(Brush, Ceiling) are unifiable by replacing lx by Brush and 
ly by Ceiling. However P{lx, ly) and ~^P(lx^ ly) are not unifiable, nor are 
P(Brush, Ceiling) and P{lx^ lx) be so. 

In the context of a plan, two literals h and I2 might not be unifiable if 
the variable-binding constraints Binding(i7) make two parameters of li and 
I 2 separate. For example, Q{lx) and Q{ly) are not unifiable if {lx ^ly) G 
Binding(iJ). If two literals pi and p 2 are unifiable by this definition, then 
we say that it is consistent that pi = P 2 , or Consistent jj(pi,P 2 )- Chapter 4 
presents a more detailed overview of unification algorithms. 

To define a negative threat, let {s^ ^ sj) be a causal link in a plan 77. 
Let Sfc be a step in the same plan such that 

1. it is consistent for Sk to be ordered between and Sj. 

That is, both Consistent jj{si\-^Sk) and Consistent jj{ski-^Sj) hold; and 

2. there exists an effect q of which can delete p. 

That is, 3q € Eff(sfc). Consistently (g — ->p). 

Then Sk is a threat to the causal link cl = (s^ Sj). The triple [d, Sk,q] is 
called a conflict. 

Similarly, Sk poses a positive threat when its presence can make 
useless. More precisely, let be a step in the same plan such that 

1. it is consistent for s/e to be ordered between and Sj. 

2. Sk can reassert p. 

That is, 3r G Eff(sfc). Consistent y^(r = p). 

One way to implement the subroutine Correct () is as follows: 

Correct (7T) returns TRUE if the precedence relation and variable-binding 
relations are consistent, if there are no more open preconditions in the plan, 
and if there are no negative threats to any causal link in the plan. However, 
this is not the only way of defining Correct () . In Section 3.4.1 we will discuss 
another correctness-checking method. 
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Fig. 2.4. (a) Demotion, (b) Promotion 



2.3.2 Threat Detection and Resolution 

Subroutine Threats-exist () can be implemented directly from the above 
definition of a threat. This subroutine takes a plan as input, and returns a set 
of (causal link, step) pairs where each step is a threat to the corresponding 
causal link. 

To remove a threat from a plan iJ, one can impose any one of the following 
constraints in the plan, as long as the resultant order and variable-binding 
relations are still consistent. Let [(s^ Sj),Sk,q] be a conflict. 

Demotion of s/^. This method corresponds to making to be ordered before 
(see Figure 2.4a). That is, add a constraint Sk^Si to the plan. 
Promotion of Sk- This method corresponds to forcing Sk to be ordered after 
Sj (see Figure 2.4b). That is, add a constraint Sji— >s/e to the plan. 
Separation. This method introduces a variable-binding constraint so that 
and q will never be the same. Suppose that for some i, Xi is the 
parameter of p and yi the parameter of q. Also suppose that either Xi 
or yi is a variable. Separation corresponds to adding a variable-binding 
constraint Xi ^ yi. This will make it impossible for p and q to clash due 
to a threat. 

Each of these threat-resolution methods can introduce one or more successor 
plans. Subroutine Resolve-Threat (t, /I) is implemented so that it returns 
the set of all successor plans as a result of resolving a threat t. 

2.3.3 Establishing Preconditions 

To achieve an open precondition p of a step s, we first scan the steps in U 
for possible candidate steps for achieving p. Let Sg be a step in II with an 
effect q such that 

1. Consistentp^(se'— »s), and 

2. Consistent jj (g = p). 
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Then Sg is a candidate establisher or producer of p for s. 

Likewise, we also search the set of planning operators in O to find can- 
didate establishers for p. Let a be an operator with an effect g, such that p 
and q are unifiable. A unifier for p and g is a set of pairs 

9 = {7x2, t2),.. ■ , (?2:n,^n)} 

where each ?Xi is a variable, and each ti is either a variable or a constant. 
Applying 9 top and g means that every occurrence of Ixi in p or g is replaced 
by the corresponding term ti. After the substitution operation, p and g look 
identical. Once the unifier is found, we can apply it to the entire operator 
schema and plan structure to obtain an instance of the operator and plan, 
respectively. For a more detailed introduction to unification, see Chapter 4. 

Based on unification, the subroutine Establish-Precondition is imple- 
mented as follows. For each candidate establisher Sg, we generate a successor 
plan as an instance of U. The set of all such instances constitutes the succes- 
sor plans for 77. More specifically, let ^ be a unifier of p and g. A successor 
is obtained as follows: 

1. apply 9 to n and Sg, 

2. if Sg is a new step, add Sg to Steps (77), 

3. add ordering constraints SgH->s and Sinit^Se to Order(77), 

4. let p' be an instance of p obtained by applying 9 to p. Add (sg ^ s) to 

C-Links(77). 

Subroutine Establish-Preconditions returns the set of all successor plans 
generated in this manner. 

Properties of Planning Algorithms, 

Two formal properties have been used most often in judging planning algo- 
rithms. A planner satisfies the Soundness Property if every solution 77 
that is output by the planner is correct An orthogonal property is the Com- 
pleteness Property^ requiring that if a solution exists for a planning problem 
( init^ goal ), then the planner will terminate with a solution (although not 
necessary the optimal one). 

Suppose that, in addition to completeness, we further require that, for 
every planning problem ( init^ goal ) that has a solution, a planner can find 
the least-cost solution. Then the planner is called admissible. 

To show that a planner is both sound and complete, a useful proof method 
is induction. For our partial-order planner POPLAN, if a least-cost-first search 
strategy is used in selecting a plan from the open list (Step 3 in Table 2.2, 
assuming all operators have positive costs) , then by induction it can be shown 
to be both sound and complete. 

Soundness and completeness, however, are often guaranteed with a heavy 
toll in search efficiency. Completeness, for example, sometimes requires that 
the planner plow through the entire search space without missing a single 
frontier. The over-cautiousness is likely to cause exponential growth in the 
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size of the search frontier, or the open-list. Several alternative methods have 
been suggested to get around this problem. One is to settle for approximate 
solutions, solutions that are sometimes incorrect (see, for example [57]) or 
highly likely to be correct [156]. Another is the infusion of strong domain 
knowledge for pruning portions of the search space that do not contain solu- 
tions. A third method is to use local-search based planning algorithms, which 
we will discuss at the end of the chapter. 

2.3.4 A House-Painting Example 

We now illustrate the Poplan algorithm with a painting example, where 
our goals are to paint a ceiling, a door and a table. To achieve these goals, 
suppose also that we have available the operators as shown in Table 2.3. In 
the initial state, we state that all brushes are dry: 

Dry(Bl), Dry(B2), Dry(B3) 

We wish to achieve the goals 

Painted (Ceiling), Painted(Door), Painted(Table) 

To begin with, a plan with a start step and a finish step is gener- 
ated (see Figure 2.5). Suppose that goals are achieved in a top-down or- 
der. Then Painted(Ceiling) is the first subgoal selected, and the operator 
paint -ceiling is converted into a step to be inserted into the plan (see 
Figure 2.6). Next an open precondition Dry(?cbr), of paint -ceiling, is se- 
lected. To achieve this subgoal a new causal link is created from the start 
step to support paint-ceiling. 

When the next subgoal Painted (Door) is achieved, a paint -door step is 
inserted into the plan, as shown in Figure 2.7. This step becomes a threat 
to the causal link (s^nit ^ paint-ceiling), where p = Dry(Bl). To resolve 



Table 2.3. Operator definition with resources 



paint-ceiling(?cbr, Ceiling) 
Preconditions Effects 



Dry(?cbr) 

paint-door(?dbr, Door) 
Preconditions 
Dry(?dbr) 

paint -t able (?tbr, Table) 
Preconditions 
Dry(?tbr) 



Painted (Ceiling) , 
-iDry(?cbr) 



Effects 

Painted(Door), 

-iDry(?dbr) 

Effects 

Painted (Table), 
-iDry(?tbr) 



Cost 

5.0 



Cost 

5.0 



Cost 

5.0 
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painted(Ceiling) 
painted(Door) 
painted(Table) 

Fig. 2.5. An initial painting plan 



V 1 

Fig. 2.6. A one-step plan for painting. The variable lehr is instantiated to B1 



( Start Step} - 



painbceiling 



painted(Ceiling) 

painted(Door) 

painted(Table) 



( Start Step} - 




?dbr 4: B1 



Fig. 2.7. A two-step plan for painting. A separation constraint is added to resolve 
a conflict 



this conflict, a separation constraint is suggested by the planner, creating an 
inequality B1 ^ ?dbr. 

When the last goal Painted (Table) is achieved, and when the two threats 
in the plan are resolved, the plan is as shown in Figure 2.8. In the final plan, 
every precondition has an associated causal link, and no threats remain. This 
plan chooses paint brush B1 to paint the ceiling, B2 to paint the door, and B3 
to paint the table. Because each operation relies on a different paint brush, 
they can be executed in parallel. 
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I ?tbr;^B1.?tbr^tB2. ?tbr=B3 j 

V / 

Fig. 2.8. A final solution plan for painting 

2.4 Total-Order, Forward- Chaining Algorithms 

A forward-chaining algorithm starts from an initial state description and 
searches for a goal in a space of domain situations (see Figure 2.9). Each 
situation is a description of the world in which an agent might find itself. 
Fully instantiated steps are added one at a time to the end of a plan, and a 
total-ordered solution plan is returned whenever a goal is reached. 

In the implementation of forward-chaining shown in Table 2.4, a node 
encodes the information along a search path. Specifically, a node is a. 
tuple: 

^fc “ (state, parent node, s, cost) 

In this representation, the state is a current state description of the agent, 
the parent node is a node which leads to the present node by one step, s. The 
cost is the total cost of the path up to the current node, possibly augmented 
with some heuristic information. For each node State-of(iV£^) returns 
the state element of that node. 

In step 3 of the ToPlan algorithm a node (and the corresponding state) 
is selected to be expanded^ resulting in a set of successor nodes to be gener- 



Direction of Plan Growth: 




Fig. 2.9. Forward search in a space of states 
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Table 2.4. The ToPlan algorithm 



Algorithm ToPlan(0, States ,State^ ) ; 

Input: A set of planning operators O, 

an initial state States and a goal state description State^ 

Output: A totally ordered, fully instantiated plan if a solution can be found. 
Otherwise, Fail. 

1 OpenList := {(State,, 0,0,0)}. 

2 repeat 

3 ^fc’~ lowest cost node in OpenList; 

4 remove from OpenList; 

5 if Statep C State-of(iV£^) then return (Solution-Path(iV£^)); 

6 else 

7 generate successor nodes of iV^^from the applicable operators in O; 

8 for each successor node Succ do 

9 if State-of(Succ) is not previously expanded then 

10 add Succ to OpenList; 

11 else 

12 compare the cost of Succ to previous path; 

13 if Succ has lower cost then 

14 add Succ to OpenList; 

15 end if (step 13) 

16 end if (step 9) 

17 end for 

18 end if (step 5) 

19 until OpenList is empty; 

20 return(Fail); 



ated. To expand a node, all operators in O are searched to find those whose 
preconditions match the state of Each instance is applied to yield a suc- 
cessor node. When a goal state is encountered, a function Solution-Path is 
applied to trace the plan steps from the goal back to the root node, returning 
the inverse of the step sequence. This sequence is a total-order solution plan. 

As with partial-order plans, by induction one can show that the ToPlan 
algorithm is both sound and complete when plans are selected from the open- 
list in a least-cost-first order. 

Conversion from Total- Order to Partial-Order Plans. A total-order plan can 
be converted to a partial-order plan by removing constraints that are not nec- 
essary for maintaining its correctness. Backstrom [15] presents an in-depth 
analysis of this conversion problem as well as of several other previous algo- 
rithms for solving this problem. He shows that obtaining the least-constrained 
version of a total-order plan is NP-hard. However, one can still employ a 
greedy algorithm to find a sub-optimal version of the partial-order plan from 
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a total-order solution. For example, one can incrementally take away the or- 
dering constraints between pairs of plan steps. A step-order Si^-^Sj can be 
removed if after the removal, the plan remains correct. One can repeat this 
procedure until no more ordering constraints can be further removed with- 
out making the plan incorrect. The plan thus obtained is a partial-order plan, 
although it may not be the least-constrained partial-order plan. 



2.5 More Advanced Operator Representations 

The above definitions and algorithms are all based on the basic Strips rep- 
resentation. Extensions to more excessive representations turned out to be 
quite natural and straightforward. In this section we consider one extension in 
planning operator representation, and discuss how the basic plan-generation 
algorithms can be extended accordingly. In the rest of the book, we always 
base our discussion on the Strips representation, mainly for its simplicity. 
The extension presented here is based on the UcPOP representation and al- 
gorithm which are designed and implemented by Barrett and Weld [105]. 

The Strips representation is sometimes called propositional, since it per- 
mits only conjunctive literals as operator preconditions and effects. In many 
practical situations, however, the propositional representation is not suffi- 
cient. One often needs to consider universally quantified preconditions and 
effects, as well as conditional effects. 

Consider the house painting example. Suppose that one requirement for 
painting the ceiling is that all furniture in the room is covered with drop- 
cloth. Instead of stating Covered(F) for each individual piece of furniture F, 
we can use a variable ?/ to denote any piece of furniture, and state 

V ?/ s.t. Inroom(?/, Room). Covered(?/) (2-1) 

In English, this sentence is read “for all ?/ such that Inroom(?/, Room) is 
true, Covered(?/) holds”. If in a room there are many pieces of furniture, 
this representation could save a lot of effort in domain encoding. Universal 
quantification could be similarly extended to operator effects. 

We could also allow operator effects to be conditional, in the form of 

effwhen cond 

This means that when cond is true, ejO^will also be true after the application of 
the operator. As an example involving both generalizations of effects, suppose 
that in the painting domain, we state a fact that after painting, all the rooms 
that are painted in a building B would smell of paint. We could say 

V ?r s.t. Inbuilding ( ?r, R). (Smells(?r) when Painted(?r)) (2.2) 

The meaning of this first-order sentence is that for all rooms ?r in a building 
B, ?r smells after it is painted. 
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Assume that all predicates hold for a finite, static set of objects. The basic 
planning algorithms are quite straightforward to extend in order to account 
for the universally quantified operator representations [105]. Suppose that 
each universally quantified variable x satisfies some predicate T, such that 
T{x) holds for a finite, static set of objects. T is the predicate Inroom in 
equation (2.1), and the predicate Inbuilding in equation (2.2). The objects Oi 
are assumed to be finite in number, and are declared in the initial state as 
T(oi), T(o 2 ), • • • , T(on). In equation (2.1) they are diff'erent pieces of furniture 
and in equation (2.2) they are the individual rooms in a building. Then for 
each universally quantified precondition of the form 

V?o; s,t. T{?x). P{?x) 

we explicitly state all P{oi) literals as a conjunction: 

P(0l),P(02),-*-,i^(0n) 

As such the Poplan algorithm need not change much in order to accommo- 
date for universal quantification. 

For conditional effects, the backward-chaining partial-order planning algo- 
rithm can be extended as follows. During planning, procedure Threat s-Exist 
looks for threats to a causal link cl = {si Sj). A step is a {negative) threat 
to cl if Sfc could possibly be between and s^, and 

(a) an effect literal of possibly denies p, or 

(b) Sfc has a conditional effect {-^eff when cond)^ where effa.nd p 

unify. 

Case (a) above could be handled by the original algorithm, while case (b) 
serves as part of the extension. 

For case (b), a method known as confrontation could be used. Specifically, 
the conditional effect states that -le^ holds when cond holds. Therefore one 
way to remove the threat is to make cond false. This could be ensured by 
adding -^cond as an open precondition of the step s^. 

Likewise, for Establish-Preconditions to handle conditional effects, 
we make the following change. To achieve a precondition p, the procedure 
searches for an operator or a step with an effect e or a conditional effect of 
the form (e when c), where e unifies with p. In the latter case, c will be 
added as a precondition of the step that establishes the new causal link for p. 

To sum up, the foregoing discussion demonstrates that the partial-order 
planning algorithm requires only minimal change when the domain language 
is extended to more expressive forms. The same claim holds for total-order 
planners. 
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2.6 Search Control 

A plan-generation process can be considered as a search in a space of plans. 
The root node of the search space corresponds to an initial plan. At any step 
of the search process, a set of successor nodes is generated to expand the 
current search horizon. Search is successful if a node is found that contains 
a correct plan. 

For both total-order and partial-order planning, the search space can be 
quite large. The problem of controlling the search is therefore very important. 
In general, the search-control problem occurs in two dimensions. In the first, 
a decision has to be made as to which node among the set of all frontier 
nodes should be selected next for expansion. A naive strategy would be to 
order the frontier nodes by the total cost of operators in a current plan; a 
plan with the minimal cost is selected next. 

A second dimension of search control is defined as the problem of selecting 
a next “flaw” to work on, given that a frontier node or a plan has been 
selected. Here a flaw is either an open precondition or a threat to be resolved. 
Extensive research in planning has shown that the order in which the flaws 
are resolved often has a profound effect on search time and space [107]. 

Following the development of UCPOP, Peot and Smith [107] suggested the 
following strategy: 

1. if there exists a threat that has zero or one resolution option, it is selected; 

2. else, if an open precondition exists, achieve the open precondition; 

3. otherwise, all threats have more than one resolution option. These threats 
are called unforced. Arbitrarily select an unenforced threat to resolve. 

This strategy is called “delay unforced threat,” or DUnf. 

A problem of the DUnf strategy is that in the last step, no preference is 
specified as to which unenforced threat to resolve next. This problem is fixed 
by Joslin and Pollack [68] in a generalized strategy called LCFR (least-cost 
flaw repair). The repair cost of an open precondition or a threat is defined to 
be the number of nodes generated as a result of repairing it. LCFR will select 
a threat with minimal repair cost. Experiments show that in many domains 
LCFR is indeed an effective heuristic to use [68]. The LCFR strategy is later 
extended by Srinivasan and Howe [127]. 

Attempting to maintain focus on the achievement of a particular open 
condition or the resolution of a threat, Schubert and Gerevini [116] proposed 
a zero-commitment, last-in first-out selection strategy (ZLIFO). Given a plan, 
the strategy works as follows: 

1. if there is a threat in the plan, select the threat to work on (however, 
delay separation). If there is more than one threat to select from, select 
the one most recently added in the plan (LIFO); 

2. if there is a unachievable or uniquely achievable open condition in the 
plan, select the condition; 
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3. otherwise, select an open precondition that is most recently added to the 
plan (LEFO). 

The strategy derives its name from the fact that Step 2 above corresponds 
to a zero-commitment choice, and that the selection method is based on the 
well-known LIFO strategy. The ZLIFO strategy was found to be very effective 
in a number of test domains. 



2.7 Satisfaction Based Methods 

2.7.1 Overview 

The above methods are called systematic planning algorithms because they 
add actions in a plan incrementally and explore the space of plans system- 
atically. In contrast, Kautz and Selman have developed a local-search based 
method for plan generation. This method is a randomized procedure for solv- 
ing satisfiability type of problems in combinatorics. It is shown to outperform 
the systematic planners in several domains. 

Generally speaking, a systematic search method starts from an empty 
plan, and repeatedly adds actions and constraints to a partially completed 
plan. The process continues until all goals are achieved and the entire plan 
becomes correct. A local-search algorithm, however, often starts from a ran- 
domly generated solution. The solution might contain some conflicts or errors 
due to unachieved preconditions or missing causal-links. A local-search pro- 
cess will then remove these errors and conflicts. In each step of the search 
process, a part of the solution is “repaired” so as to minimize the number of 
errors in a new solution. 

Central to the local search approach to planning is the ability to form a 
solution, although the solution can be incorrect. However, before planning is 
completely done, there is usually no solution for one to work with; among 
other things, we do not know how many steps will be required in the final 
solution plan. Kautz and Selman solved the problem by using a clever idea, 
derived from Blum and Furst’s Graphplan [21]. They consider the world 
as progressing through a sequence of time points t = 0, 1, 2, . . . , fc. Each time 
point gives rise to a state, which is a completely specified and instantiated set 
of propositions. Then, assuming that the plan would take n steps for a fixed 
integer n, an attempt is made to verify that a correct plan of n steps can be 
found, leading from the initial state to the state at time point n. The plan 
should achieve the goal formula no later than the last time point n. If no such 
plan is found within a certain upper bound on the amount of computation, 
n is incremented and the local-search process repeated. 

To implement the above idea, three key elements are needed. First, a 
method for specifying what a correct solution plan is must be designed. This 
specification is also known as a domain models describing the consistency 
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constraints and actions for a state. The domain model is indexed on time 
point t. Second, a method must be designed for first instantiating the do- 
main model at every time point up to and including time point n, and then 
converting the domain model to a uniform representation that is acceptable 
by a local search algorithm. A local search algorithm (for example, Gsat; see 
Chapter 4) is then activated. This step is also known as model satisfaction. If 
successful, model satisfaction produces a sequence of states. Finally, the se- 
quence of states are converted to a sequence of actions. This action sequence 
corresponds to a solution plan. 



2.7.2 Model Definition 

To motivate the satisfaction-based methods, consider an example of moving a 
book from someone’s home to a university. We can describe this domain using 
the following predicates. Connected(loca, locb) means that locations loca 
and locb are connected. We do not include a time index for Connected be- 
cause it does not change in different states. An object can be moved between 
any two connected locations. At(object, loc, z) means that at time point z, 
object object is located at location loc. Inside( object, container, z) means 
that at time point z, object is inside container. 

Following a state-based encoding of actions, we first describe the facts that 
hold in each state at every time point (thus all formulas below are universally 
quantified). 

— no object can he at two different locations at the same time. 

(At(object, loca, z)A(loca ^ locb))=^-iAt(object, locb, z) 

— no object can he at a location and inside a container at the same time. 

-i(At(object, loca, z)Alnside( object, container, z)) 

— Home and University are always connected. 

Connected(Home, University) 

— no object can be inside two different containers at the same time. 

-i(Inside( object, containera, z)Alnside(object, container^, z)A 
(containera ^ container^)) 

Next, we describe all ways in which a literal “flows” from one time point 
to the next, and we do this for all literals in the domain: 

— if an object is at a location, it either remains at that location or goes into a 
container that is also at that location. The latter possibility describes the 
action “load object into a container.” 
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At(object, loc, i)=^ 

At(object, loc, i 4- 1)V 
3 container G {SchoolBag, BriefCase}. 

(At (container, loc, i)A 
Inside(object, container, i + 1)) 

— if a container is at a location, it either remains at that location or is trans- 
ported to a connected location. The latter possibility describes the action 
“move container to another location.” 

At(container, loc, i)=^ 

At(container, loc, i + 1)V 
3locb € {Home, University}. 

(Connected(loc, locb)A 
At(container, locb, i + 1)) 

— if an object is in a container, it either remains in that container or is located 
at the same location as the container. The latter possibility describes the 
action “unload object from container.” 

Inside(object, container, i)=^ 

Inside(object, container, i + 1)V 
Bloc € (Home, University}. 

(At (container, loc, i)A 
At(object, loc, i + 1)) 

The above model of the domain constrains the truth values of various 
propositions. For example, if an object is placed inside a container, the second 
state axiom makes sure that the object is no longer at its original location. 
This constraint ensures that the object follows the container wherever the 
latter moves. 

The initial state includes all facts true at time point 0. For the above 
example, these facts are: At (SchoolBag, Home, 0), At(BriefCcise, Home, 0), 
At(Book, Home, 0) and Connected(Home, University). 

Similarly, the goal is a logical formula which states what is expected to be 
true at the last time point. It can also be a statement of a constraint on the 
behavior of the agent; for example, a goal might state that a certain formula 
must be true at the second to last time point. The last time point is of course 
unknown before the problem is solved. So we can initially set the point to 0 
and then increment it repeatedly. For the book-carrying problem above, we 
could set it to be at the last time point n: 

goal = At (Book, University, n). 

We can then systematically push the envelope defined by n from 0 onward, 
and verify the logical model at each intermediate time point. 
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2.7.3 Model Satisfaction and Plan Construction 

To verify that a model exists up to and including time point i using local 
search algorithms, we must convert the logical model to a form acceptable 
by the search algorithm. 

First, each universally quantified variable such as loc and time points i are 
instantiated to constants. For example, a formula of the form V i. At(A, J5, i) 
is instantiated into 

At(A, B, 0)AAt(A, B, l)AAt(A, 2) A . . . At(A, n) 

The above axioms can be converted to a logical form known as conjunc- 
tive normal form; the conversion is straightforward (see [102], p. 145 for an 
algorithm). After this conversion we obtain a set of clauses, each clause being 
a set of literals that are fully instantiated. The set of clauses represents the 
conjunction of the clauses in the set, and each clause represents a disjunction 
of the literals in it. 

Let the set of clauses representing the entire model at time point i be 
CSet(z). Let the set of all clause sets up to and including time point n be 
S{n): 

Z{n) = {CSet(0)uCSet(l)U . . . CSet(n)} 

We can then invoke the SatPlan algorithm described in Table 2.5. In this 
algorithm, the input parameter MaxEnv represents the maximum number of 
plan steps we are willing to examine. 



Table 2.5. The SatPlan algorithm 



Algorithm SatPlan(); 

Input: A set clauses CSet(i), a parameter MaxEnv 
Output: a sequence of states if successful; Fail otherwise. 



1 for n := 0 to MaxEnv do 

2 decide the maximum number of tries and flips; 

3 call GSAT(I7(n)) (Gsat is described in Chapter 4); 

4 if Gsat returns a satisfying assignment T then 

5 convert T to a sequence of states 5; 

6 determine actions that go between the states (see below); 
let resulting plan be iJ; 

7 return 77; 

8 end if 

9 end for 

10 return(Fail); 
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Fig. 2.10. SatPlan solves a problem by verifying models on a timeline 



There are two possible outcomes for the algorithm: either the model is 
satisfiable within a specified time bound, or it is not. In the former case, we 
are provided with a sequence of states, starting from the time point 0 and 
ending at e, where e is the last time point for the particular model. The next 
task is to construct an action sequence from the state sequence. 

For the book-carrying example the time points in a satisfied model are 
shown in Figure 2.10. Corresponding to each time point is a state: 



stateo 



f At(Book, Home, 0), At(SchoolBag, Home, 0) 

^ At(BriefCase, Home, 0) , Connected(Home, University) 



statei = 



state2 = 



Inside(Book, SchoolBag, 1), 
At(BriefCase, Home, 1), 

Inside(Book, SchoolBag, 2), 
At(BriefCase, Home, 2), 



At (SchoolBag, Home, 1) 
Connected (Home, University) 

At(SchoolBag, University, 2) 
Connected(Home, University) 



states 



At(Book, University, 3) , At(SchoolBag, University, 3) 
At (BriefCase, Home, 3) , Connected (Home, University) 



From the above sequence of states, we could obtain a set of adjacent states 



(stateo, statei), (statei, state2), (state2, states). 

Each pair can then be used to select an action responsible for the transition. 
This can be done by applying actions, one at a time, to the initiating state, 
and comparing the resulting state with the second element in each pair. If 
the resulting state corresponds to the second element, the action is chosen. 
To do this, however, requires that we have a representation of the individual 
actions. For the book-carrying domain, our actions are listed below. 

load : 

At(object, loc, i)A 
At(container, loc, i)=^ 

Inside(object, container, i 4- 1) 
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unload : 

Inside(object, container, i)A 
At(container, loc, 

At(object, loc, i + 1) 

move : 

At(container, loca, i)A 
Connected(loca, locb)=^ 

At(container, locb, i + 1) 

After the computation is done, we end up with a plan: 
load(Book, SchoolBag, 0) 

i 

move (SchoolBag, Home, University, 1) 

i 

unload(Book, SchoolBag, 2) 

Kautz and Selman report that for large-scale blocks- world problems and 
a logistics planning problem, SatPlan is several orders of magnitude faster 
than planning algorithms that are based on systematic search methods [77]. 
In the same set of experiments, several other variations of GSAT are applied, 
including the Davis-Putnam procedure developed by Crawford and Auton 
[33] and the Walksat algorithm [120]. 



2.8 Background 

The domain and operator language in this chapter are adapted from Chap- 
man’s Tweak [28], which uses a version of Strips domain language [46]. 
The PoPLAN algorithm with its partial-order plan representation is based on 
the Snip algorithm [94, 17]. The ToPlan algorithm is a simplified version 
of the A* algorithm described in most AI textbooks [102]. 

How to implement a correctness-checking algorithm for a partial-order 
plan has been a subject of ongoing investigation. Chapman defines a modal- 
truth criterion for TWEAK [28]. The criterion was considerably simplified in 
an implementation of AbTweak, a hierarchical version of Tweak [159, 149, 
160] . The Snip algorithm [94, 17] introduces another level of simplification 
in Chapman’s criterion, by eliminating the so-called white knights altogether 
(see Chapter 7). A thorough discussion can be found in [75]. Kambhampati 
presents an extension to a backward-chaining, partial-order planner, using 
multi-contributor causal structures [71]. Veloso and Bl 3 rthe discuss the need 
for combining total-order and partial-order planning in a single framework 
[139]. Also see [129] for a discussion for the need for different heuristics to 
guide a planner. Joslin and Pollack distinguish between active and passive 
commitments in constraint posting, and propose as well as evaluate a tech- 
nique for representing and reasoning about all planning decisions with con- 
straints [69]. 
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The more elaborate operator language in Section 2.5 is based on the 
UCPOP formalism [105], which is a partial-order implementation of Ped- 
nault’s ADL language for planning domain description [104]. For total-order 
planners, the extension can be done similarly. See, for example, [10, 19] for 
a description of the TlPlan planner, which encodes domain knowledge for 
controlling a total-order, forward-chaining planner. 

Finally, there has been a new thrust in planning with non-STRlPS style 
action encoding. Examples are Graphplan [21] and local-search based meth- 
ods [77]. SatPlan, described in the last section, is adapted from an AAAI 
96 paper by Kautz and Selman [77]. Due to space limitations, we only in- 
clude one form of state-based encodings in which no explicit propositions for 
actions are used. Other forms of encodings that include actions as a form 
of propositions have been attempted. It should be noted that so far no one 
approach yet appears best for all domains. 
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3.1 Basic Analytical Techniques 

Plan generation can be viewed as a search in a space of nodes. This is true for 
both forward-chaining and backward chaining planners, and for total-order 
and partial-order planners. 

Consider the two planners that we discussed in the previous chapter. For 
a forward-chaining planner, a node records state information, and the arcs 
connecting the states are formed by operators applicable to the originating 
states. For a backward-chaining planner a node is a plan, and the arcs con- 
necting the nodes are plan-modification operations applicable to a plan. In 
both cases, a planner explores a tree of nodes. This is known as the search 
tree of a planner. 

In a search tree, the number of nodes, together with the amount of pro- 
cessing time on an individual node, give rise to an estimation of the time 
complexity. We number the level of nodes in a search tree starting from the 
root node; the root has a level number zero. The children nodes of the root 
are at level one, and so on. For each node at level i of the search tree, let 
Bi be the maximum number of arcs originating from the node. Let D be the 
minimum number of arcs explored by the planner on a path leading from the 
root (see Figure 3.1). D could be used to measure the search-tree depth under 



Root Node 




Fig. 3.1. A search tree explored by a planner 
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the assumption that the planner will not wander beyond this depth during a 
search process. Also let be the maximum amount of time spent by the 

planner on each node. Then the worst-case time complexity of a total-order 
or partial-order planner could be bounded by 

D i 

Time(PLANNER) = ^((J^ * T^ode) (3-1) 

i=0 j=0 

To estimate the time complexity of a specific planner, we must flesh out three 
factors, Bi, D and 

3.2 Analyzing Forward-Chaining, Total-Order Planners 

The branching factor of ToPlan is determined by the number of applicable 
operator instances to each state. The effective branching factor Bj: is set to be 
the maximum such number. Notice that the number of applicable operator in- 
stances could be much larger than the number of planning operator templates 
themselves, since each planning operator template could be instantiated to. 
many instances. As ah example, consider planning a path in a building from 
one room to another. The domain might have only one operator template 
for moving. But for each room, the number of instances is determined by the 
number of adjacent rooms. 

The search depth of ToPlan is determined by the minimum number of 
plan steps leading from the root node to an optimal goal, which we denote 
by N. Figure 3.2 shows the parameters. 

The time spent on each node is determined by the time needed to unify 
an operator’s preconditions to the literals in a state, and by the time spent 



Initial State 




Goal State 



Fig. 3.2. Search tree for a forward, total-order planner 
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to generate successor states and checking whether these states have been 
generated before. This time, Tp is assumed constant throughout the entire 
search space. 

Putting it together, we obtain a worst-case time-complexity formula for 
forward, total-order search: 

Time(ToPLAN) = 0{bJ^ * Tj) (3.2) 

The formula points out one potential shortcoming of forward, total-order 
search. The branching factor is determined by the number of applicable 
operators to each state, and this number could be extremely large. To see 
this, consider building a wooden table from a pile of lumber. Each applicable 
tool that could change the state of wood, ranging from drills and saws to 
hammers and nails, can potentially give rise to a large number of applicable 
operators. The branching factor in this case could even be infinite. 

It is due to shortcomings like this that early AI researchers have turned 
away from pure forward-chaining planners. An alternative is backward- 
chaining planners. In these planners, the branching factor is determined by 
the number of ways to achieve a goal. This number could be potentially much 
smaller than the number of operators applicable to a state; in general, the 
number of ways to make a hole in a piece of wood is smaller than the number 
of ways to shape the same piece of wood. We will analyze backward planners 
in more detail in the next section. 



3.3 Analyzing Partial-Order Planners 

Recall that in the POPLAN algorithm, described in Chapter 2, the following 
components need be specified: 

step 5. Correct (iJ) , as a termination condition; 

steps 7 &: 8. Threat s-Exist and Re solve -Threat, for detecting and resolv- 
ing threats in a plan; 

step 10. Establish-Precondition, for building a causal link for an open 
precondition. 

In the PoPLAN algorithm, B is the maximum number of successor plans 
generated either after step 8, or after step 10. The depth D is the maximum 
number of plan expansions in the search tree from the initial plan state to 
the solution plan state. Let P denote the maximum number of preconditions 
or effects for a single step, and let N denote the total number of plan steps 
in an optimal solution plan (except the start and finish steps). 

To fiesh out the effective branching factor R, we first define the following 
additional parameters. We use Bnew for the number of new operators found 
by step 10 for achieving p. Bold for the number of existing plan steps found 
by step 10 for achieving p, and r for the number of alternative constraints for 
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removing one threat. The eflPective branching factor of search by the POPLAN 
algorithm is either Bnew + Boidi oi r since each time the main routine is 
followed, either step 8 is executed for removing threats, or step 10 is executed 
to build causal links. If step 8 is executed, r successor states are generated. 
Otherwise, ( Bnew + Bold) successor plan states are generated. 

Next, we expand the effective depth D. In a solution plan, there are N^P 
number of {p^s need) pairs, where p is a precondition for step Sneed- Let / be 
the fraction of the N * P pairs chosen by step 10. Let v be the total number 
of times any fixed pair (p, Sneed) is chosen by step 10. Finally, for each pair 
{Pj Sneed) to be achieved, a set of threats is accumulated to be removed. Let t 
be the number of threats generated by step 10. A summary of the parameters 
can be found in Table 3.1. 

The total time-complexity formula can now be obtained. Since the number 
of nodes in a tree is roughly the number of leaf nodes, we count the number of 
leaves in the search tree (see Figure 3.3). Step 10 for achieving open precon- 
ditions is executed / * AT * P * u times, each time generating ( Bnew + Bold) 
successor plan states. For each of these states, step 8 for resolving threats 
is executed t times, each time generating an additional r successor plans. 
Putting these together in equation (3.1), we get 

#nodes(POPLAN) = 



To obtain the time complexity, we must determine the time complexity for 
evaluating the termination condition. Correct (il). For illustrative purposes, 
we consider the following version of the termination condition: 

1. every precondition of every step must have a causal link, 

2. no negative threats exist for any causal link. 



f*N*P^v i 

E 

i=0 j=0 

O (( * r 

O ([(B„e,. + Bold) * 



+ Bold) * 



(3.3) 



Table 3.1. Summary of notations 



B effective branching factor 

D effective search depth 

Tnode average time per node 
N total number of operators in a plan 

P total number of preconditions per operator 

/ fraction of (p, Sneed) pairs examined 

V average number of times a (p, Sneed) pair is visited 

t average number of threats foimd at each node 

r average number of ways to resolve a threat 

Bnew average number of new estabhshers for a precondition 
Bold average number of existing (or old) establishers for a precondition 
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Fig. 3.3. Search tree for a backward, partial-order planner 



To check this condition efficiently, one can maintain a list of threats and 
open preconditions. When this list is empty, the termination condition is 
true. Thus, the time per node is constant, or = 0(1). 

Also, to establish causal links, every precondition in a solution plan must 
be visited once. Thus, / = 1. In addition, since every precondition p of every 
step Sneed h^s an associated causal link, (p, Sneed) needs not be visited more 
than once. Therefore, v = 1. For this version of planner, the time complexity 
for checking termination condition is N ^ P. Let the branching factor be 

= [( Bnew + Bold ) * (3.4) 

And the time complexity for Poplan is 

Time(PoPLAN) = O (3.5) 

We now consider a comparison of this formula with the time complexity 
for a forward-chaining total-order planner (formula (3.2)). In both planners, 
the effective search depths are the same. To make the comparison simple, 
we also assume that the time-per-node factor Ty is a constant. A total-order 
planner will take more time to find a solution if its branching factor Bj: is 
larger than B^ for a backward-chaining, partial-order planner. This is the 
situation in a domain where the number of ways to modify a state is large 
(hence Bj: is large), and the factor is small. The latter is true when the 
number of ways to achieve a goal is small (hence Bnew and Bold are small), 
and the number of ways to resolve the threats in a plan is small (hence r is 
small). On the other hand, a forward-chaining total-order planner will take 
less time than a backward-chaining, partial-order one when the number of 
operators orginating from a state is small, and the number of ways to achieve 
a goal and to resolve threats in a plan are large. 
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3.4 Case Study: An Analysis of Tweak 

Instead of building explicit causal links for each precondition, Chapman’s 
Tweak [28] uses what is known as the Modal Truth Criterion to verify the 
correctness of a plan. In this section we describe a simplified implementation 
of Tweak, to which we apply the analytical formula derived in the last 
section. This simplified planner is the basis for an abstract version of Tweak 
known as AbTweak[160]. 

3.4.1 An Implementation of Tweak 

We focus on an implementation of the modal truth criterion. Consider a pre- 
condition p for a plan step Sneed- P is called correct if the following conditions 
are true: 

Establisher. There exists a plan step 5add such that Sadd is an establisher of p 
for Sneed- To qualify as an establisher, Sadd must be ordered before Sneed 
in 77, and must have an effect p. 

Conflict-Free. All clobbering steps are taken care of. 

A step Sk qualifies as a clobbering step if it is not ordered after Sneed? and 
one of its effects q could unify with -ip. By this definition, all negative 
threats are clobbering steps. 

To take care of a clobbering step for the pair (p, Sneed)? one of the following 
must be true. Let Sadd be an establisher of p for Sneed- 

Demotion of s^. is ordered after Sneed- That is, a constraint Sneed^^k 
exists in the plan. 

Promotion of s^. Sfc is ordered before Sadd- That is, Sfci-^5add in the plan. 
Separation. Suppose that for some i, ?Xi is the parameter of p and Ipi 
the parameter of q. Also suppose that either ?Xi or ?pi is a variable. 
Separation corresponds to a variable binding constraint ?Xi This 

will make it impossible for p and q to clash, by denying each other. 
White Knight. Let be a plan step such that it is consistent that s^t,fcH-)-5need 
and Furthermore, bas an effect e such that e and p unify. A 

white knight step is one which provides a shield between the step 5 need 
and the clobbering step s^; whenever p is in danger, 8^;^ comes to its 
rescue. 

More precisely, the white knight constraints are 

^wk^ ^^need? 

and 

— either p or ~^q is an effect of 

The first three constraints are familiar to us by now. The fourth one, the 
white knight constraint, deserves a little more explanation. 

White knight is a concept formalized by Chapman. It is plausible that 
a plan step s in a plan might be able to achieve a precondition of another 
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Introducing 
White Knight Swk 




Fig. 3.4. Resolving a conflict by introducing a white knight step Swk 



step in the plan. In such cases, and when a threat is discovered, it is possible 
to impose further ordering and variable-binding constraints onto the plans, 
such that the step s serves as a shield for the threatened precondition, against 
the clobberer. This is the process of introducing white knights, shown in 
Figure 3.4. 

It should be pointed out that it is necessary to include white knights for 
conflict resolution in a planner. A partial-order causal-link planner such as 
Snlp which does not use white knights is known to be complete; it can find 
a solution to a problem if there is one. A white knight simply provides a 
combination of promotion, demotion, separation, and precondition establish- 
ment methods used in Snlp [94, 17]. Although it is not necessary for ensuring 
completeness, it does provide a heuristic advantage in some circumstances. 
One reason why it is used in Tweak is because it provides a kind of short 
cut in a search tree; instead of imposing constraints one at a time in each 
search node, a group of constraints are imposed all at once. It is also true 
that including the white knight in search increases the branching factor of 
search by one. Therefore, whether a planning method using white knights 
will be more efficient than one not using it depends on the balance between 
reducing the path lengths along some search-tree branches and increasing the 
branching factor. 

Let our implementation of the modal truth criterion be referred to as 
MTC. Given a plan, M.T C{plan) returns TRUE if all preconditions of all 
plan steps in II are correct. If so then Tweak can terminate (Step 5 in the 
PoPLAN algorithm. Table 2.2). Otherwise, some plan step Sneed must exist 
for which the above conditions are false. In that case, MTC (iJ) can return a 
{p, -Sneed) pair. This is the subgoal to be established in step 10 of the PoPLAN 
algorithm. The rest of the algorithm could remain the same. 
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3.4.2 Analyzing Tweak 

Suppose that for each plan step, the number of preconditions and the number 
of effects are roughly the same. For a given plan il, the MTC checks every 
precondition of every plan step. This would take 0{N * P) in the worst case. 
For each precondition, the white knight part of the MTC requires that a 
search be made for clobberers, and for each clobber, another search for white 
knights. Thus, for each precondition, checking the MTC conditions takes 
0{N * P)^ time. This is the factor in equation (3.1). 

Since the same precondition might be clobbered more than once, 
u (Tweak), the average number of times a (p, Sneed) pair is visited, is greater 
than one. Also, not every precondition will be selected to achieve; if p is al- 
ready true by virtue of the initial state, for example, then it might be the 
case that p will be left alone. Therefore, /(Tweak) < 1. 

Putting this together, the time complexity for Tweak is as follows: Let 
Pb be [Priew “b Bold ) * • 

Time(TwEAK) = O * {N * (3.6) 

Judging from the analytical formulas for the causal-link planner POPLAN 
and the non-causal-link Tweak, we see that neither is absolutely better than 
the other. From equation (3.6) it appears that Tweak spends more time on 
each plan. However, because of the / and v factors, the search depth of 
Tweak may turn out to be less than N. This happens when / <C 1 and 
= 1, a situation where many of the goals and preconditions need not be 
explicitly established. In contrast, PoPLAN has a smaller processing time on 
each plan. Thus, in other domains PoPLAN can be better than Tweak. 



3.5 Critics of Basic Planners 

Comparing the worst-case behavior of ToPlan and PoPLAN, we notice that 
neither is a clear winner. While ToPlan has a potentially large branching 
factor, the search depth is fixed at AT, the total number of operators in an 
optimal plan. Poplan, on the other hand, has a depth that is P times larger 
than that of ToPlan, where P is the maximum number of preconditions for 
a plan step. In addition, although PoPLAN has a bounded branching factor, 
it is determined not only by the number of existing and new establishers 
{Bnew and Bold), but also by the number of threats t and the number of 
ways to resolve a threat r. When the number of preconditions for each plan 
step and the threats are large, the effective branching factor for partial-order 
planning could also be extremely large! 

Despite the above negative conclusion, the original intuitions of partial- 
order planning still remain appealing. A partial-order planner works on one 
subgoal at a time. This is essentially an implementation of the well-known 
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divide- and- conquer strategy to problem-solving. One impediment to this im- 
plementation is that individual solution plans interact, which in turn increases 
the number of threats considered (the factor t in time complexity formula). 

Another valuable intuition of partial-order planners is that the steps in a 
plan need not be prematurely sequenced; if most of the steps of a solution 
plan will eventually be found to be independent of each other, a situation 
where most plan steps are commutative, then it is indeed better to leave 
them unordered. 

In the rest of the book, we plan to capture the intuitions of divide- and- 
conquer under a diflPerent implementation from the one used by basic planning 
algorithms, an implementation we refer to as problem decomposition. In this 
process, a complex problem is first decomposed into several more-or-less inde- 
pendent parts. Each part is then planned for separately, in accordance with 
the divide-and-conquer strategy. For each subproblem, a forward-chaining 
planner or a partial-order planner could be used. If the former, the decom- 
posed domain structure will offer an additional advantage, by reducing the 
number of applicable operators to each state within each subproblem. This 
helps reduce the forward-search branching factor. In addition, the solution 
length for each subproblem is also reduced. This leads to a reduction of search 
depth in both forward and backward planners. Finally, solutions are com- 
bined in such a way that most of the partial-order structures are maintained 
as much as possible. This policy will most likely lead to increased compact- 
ness of solution plans since most of the plan components are commutative. 
Orders are imposed upon plan steps of solutions to different subproblems 
only when absolutely necessary. In this manner, partial- ordering is a natural 
by-product of the divide-and-conquer strategy. 

In addition to the divide-and-conquer strategy, planning at different lev- 
els of importance is another often-used approach. Distinguishing between 
important and unimportant open preconditions and threats entails that at 
any time a planner need only work on a small set of tasks, and that what 
has been achieved at higher levels of abstraction could be usefully employed 
to constrain search at lower levels. For both forward and backward chain- 
ing planners, this method of hierarchical abstraction could lead to reduced 
branching factors and depths. 

Based on the above motivation, in Parts II and III we will present planning 
algorithms that are based on divide-and-conquer and hierarchical abstraction 
and approximation in two separate parts. 



3.6 Background 

Early work on analyzing the time complexity of forward-chaining, total-order 
planners was done by Korf [83]. Starting in late 1980s researchers began 
to compare different versions of backward-chaining planners, in particular 
partial-order versus total-order planners. Minton, Drummond, and Bresina 
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[95] presented one such analysis, showing that an instance of a partial-order 
planner can be more efficient than its backward-chaining, total-order coun- 
terpart, although not necessarily always so. Barrett and Weld [17] gave an 
in-depth comparison of a wide variety of partial-order planners and total- 
order planners, both being backward-chaining ones, demonstrating that in 
general the former is more efficient than the latter. 

Work has also been done on comparing versions of partial-order planners. 
When Snlp was first presented at the AAAI91 conference, McAllister claimed 
that it was more efficient than a previous planner Tweak, because, due to 
its use of causal-link protection, it would never generate the same plan twice. 
This absolute efficiency claim turned out to be fiawed, as was shown later 
analytically and empirically by two independent groups, Knoblock and Yang 
[81] and Kambhampati [72]. These two works were later combined into a joint 
paper [74]. The analysis for partial-order planners in this chapter is adapted 
from a paper co-authored with Craig Knoblock [81], which won the best paper 
award at the 1994 Canadian AI Conference. 

The cautious protection of causal-links did have a very important side 
effect, which turned out to be the cornerstone for subsequent representational 
extensions of planning domain languages, mainly developed at University of 
Washington, Seattle. The side effect is that due to the cautious protection 
of causal links by Snlp, the amount of ambiguity within a single plan is 
drastically reduced. For example, for every precondition it is clear which 
plan step is responsible for achieving it. Work in this direction includes [105], 
[106], [85], and [45]. 

Bacchus and Kabanza performed an empirical comparison of a forward- 
chaining, total-order planner and backward-chaining, partial-order planners 
[10]. They argued that with the former it is much easier to encode domain 
knowledge for pruning large portions of the search space. This work is pre- 
dated by work done by Currie and Drummond [39], who designed a method 
for controlling search with a backward-chaining, partial-order planner. It was 
reported in [157] that with partial-order planners, Currie and Drummond’s 
method is not always a clear winner. Peot and Smith [107] and Joslin and 
Pollack [68] presented thorough evaluations of the effects of delaying the re- 
moval of threats on planning efficiency. The quest for better heuristics for 
guiding partial-order planners is ongoing. 
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4.1 Propositional Unification Algorithm 

A key component of a planning algorithm is deciding whether two literals li 
and I 2 can be made identical. Recall that a literal is either a proposition such 
as p(ti, ^2 5 • • • 5 tn) or its negation. Each parameter known as a term, of a 
literal can either be a variable Ixi or a constant Q. The literals p(?x, C\) 
and p(C2, ?y) can be made identical by substituting the variable ?x by C2 
and ?p by Ci. However, the literals p(?x, Ci) and q{^u) cannot be made 
identical, nor can p(?x, Ci) and ~^p{lx^ Ci). 

Substitution. If two literals can be made identical, they are called unifiable. 
To identify whether two literals are unifiable requires the introduction of 
substitution. A substitution can be defined as a set of pairs 

e = {{?Xi,ti), i?X2,t2), ■■■, {?Xn,tn)} 

where each ?Xi is a variable, and each term ti is either a variable or a constant. 

By applying a substitution ^ to a literal I we mean that every occurrence 
of ?Xi in I is simultaneously replaced by for i = 1, 2 , ... n. The word “si- 
multaneous” refers to the requirement that each replacement is done only 
once, the replacements for different terms are separately done, and the sub- 
stitution process is not recursive. The result is denoted by For example, 
for a literal I = p(?x, ?p, ?z) and substitution 6 = {(?x, ?z), (?p, ?x), {Iz, C)}, 
W — p{?z, ?x, C). W is called an instance of 1. 

Composition. Two substitutions can be composed together. 

Let 6 = {{?Xi,ti),i = 1, . . . n} and a = = 1, ... A:} be two substi- 

tutions. Composition(^, a) denotes a substitution 7 such that for any literal 
I, {I0)a = Ij. The composition 7 is produced by the following steps: 

a) produce a substitution 71 = {{7xi,tia),i = 1, . . .n}, by applying 
ai to every term ti] 

b) remove all identities of the form (?Xi, ?Xi) from 71, giving rise to 
72; 

c) for each variable pj, j = 1, . . . Ar, if pj does not appear in {x^,? = 

1,2, ...n}, add {?yj,Uj) to 72; 

d) return the resultant substitution 7. 
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Unification. For two literals, l\ and l 2 ^ there may be a substitution 0 such 
that li6 is identical to hO- 9 is called a unifier of li and l 2 - If no such 9 exists, 
then we say that l\ and I 2 cannot be unified. 

We consider unification in the context of a plan. Let iJ be a plan; il 
consists of a set of constraints on variable binding. l\ and I 2 can be unified 
only when these constraints are not violated. 

To find a unifier for the two literals, we could first test to see if they have 
the same number of parameters and the same propositional symbol (that is, 
the symbol p in p{lx)). If these tests are passed, we could then scan the two 
literals from left to right, building a unifier 9 along the way. Initially, 9 is an 
empty set. At the i^^ step, the i^^ parameters of l\ and I 2 are simultaneously 
extracted. Let them be u and v respectively, u and v could fall under one of 
the following cases: 

Case 1: u and v are identical. In this case we move on to examine the next 
pair of parameters. 

Case 2: u and v are different constants. In this case the two literals cannot 
be unified. The unification algorithm returns Fail. 

Case 3: At least one of u and is a variable, and u and v are constrained 
in the plan iJ by a separation constraint: u ^ v. In this case l\ and I 2 
cannot be unified either, and the algorithm returns Fail. 

Case 4: At least one of u and t; is a variable, and u and v are not constrained 
by a separation constraint. Let u be the variable. Form a substitution 
{(u,u)}. 9 is updated by 

9 := Composition( 6>, {(u,u)} 

The algorithm P-UNIFY is fully described in Table 4.1. 

Instantiating Steps, Operators and Plans. Unification is often performed 
when an open precondition is achieved. Let p be a precondition literal of 
a plan step Si^eed- Let e be an effect literal of an operator or another plan 
step s. Suppose that p and e are unifiable, and let 9 be P-UNlFY(p, e, iJ). 
An instance of the plan can then be obtained by applying 9 to all elements 
of the plan II. This means that for all pairs {lx, t) in 9, for all occurrences 
of lx in iJ, including plan steps, causal links, and separation constraints, lx 
is replaced by t. If we decide to add the new step s to the plan, we must 
first apply 9 to all elements of s as well. The resultant plan step is called an 
instance of the originating step or operator. A plan in which all variables are 
substituted by constants is called a fully instantiated plan. 

In a plan, all variable binding constraints of the form lx =ly, forcing 
lx and ly to be bound to the same constant, are implemented as actual 
substitutions: all occurrences of lx are replaced by ly, or vice versa. Thus, 
the only type of variable-binding constraints actually maintained in a plan is 
separation constraints. 
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Table 4.1. The Poplan algorithm 



Algorithm P-unify(/i, h, II); 

Input: Two literals h and and a plan il; 

Output: A unifier 9 if h and I 2 are unifiable. Fail otherwise. 



1 if one of h and I 2 is negated and the other is not, or 

1 1 and h have different propositional symbols, or 
1 1 and I 2 have different number of parameters 

2 then return(Fail); 

3 0 := 0; 

4 let n be the number of parameters in h; 

5 for i = 1 to n do 

6 let u be the parameter of /i, 

7 let V be the parameter of h; 

8 if u ^ V then 

9 if one of u and 1 ; is a variable, (say, u), and 

(u ^ v) ^ Binding(iJ) 
then 

10 9 := Composition(^, {(u,-!;))}; 

11 li := li9; I 2 := 129; 

12 if/i=/2 

13 then return 9; 

14 end if (step 12); 

15 else 

16 return(Fail); 

17 end if (step 9) 

18 end if (step 8) 

19 end for (step 5) 

20 return(^); 



4.2 Graph Algorithms 

In the process of developing a plan, many queries regarding ordering con- 
straints need be answered. These queries arise in a variety of circumstances; 
each of them requires a supporting graph-connectivity algorithm: 

Cycle Detection: is the plan a true partial-order? That is, is there a cycle in 
the plan? 

Precedence Relation: find the set of all steps which could be ordered before 
a certain step, without violating the partial-order relation. 

Topological Sort: generate a total-order consistent with a partial-order. 

In addition, later we describe two algorithms, Alpine [80] and HiPOiNT [13], 
for generating abstraction hierarchies. These algorithms depend on a graph 
algorithm for detecting strongly connected components. A strongly connected 
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Fig. 4.1. A plan and its associated transitive closure matrix 



component in a graph is a set of vertices such that for every pair of vertices 
u and u, there is a path from u to v and one from v to u. 

In this section, we review relevant graph connectivity algorithms to handle 
these queries. We assume that all steps in a plan il are uniquely numbered 
such that the numbering for a step is z. We also assume that the ordering 
information is represented using a matrix C* . For any two steps and s^, 
C*{i^j) = 1 if in plan II. Otherwise C*{i^j) = 0. C* is called a 

transitive closure. An example plan and its corresponding transitive-closure 
matrix are shown in Figure 4.1. 

Suppose that a plan il has n — 1 steps, and C is the (n — 1) x (n — 1) 
matrix representing the step-connectivity information. Now suppose that we 
added a new step s^. C could be updated by adding a new row and a new 
column, such that C{i,n) = 1 if Si^Sn and C(n,i) = 1 if The 

transitive closure could then be obtained by executing Warsh all’s algorithm 
(see Table 4.2). 

With the transitive closure matrix C*, various queries could be easily 
answered: 

Cycle Detection: The plan steps form a cycle if for some i, = 1. 

Precedence Relation: A step is before Sj in a plan if and only if C*(i, j) = 
1. Likewise, an ordering constraint Si^Sj is consistent with the existing 
partial order of a plan if C*{j, i) = 0. 

Based on the connectivity matrix , we can also implement a depth-first 
search algorithm, as shown in Table 4.3. This algorithm performs a depth-first 
search starting with a vertex i. We associate each vertex with a mark, which 
initially is set to zero. We then systematically scan the vertices beginning 
with i. To perform depth- first search on a plan, we can place a function call 
DepthFirst(O) where 0 is the index of the start step 

With the depth-first search algorithm, topological search can be easily 
implemented, whereby a total order of plan steps consistent with the existing 
partial order can be generated. Right after each step-index i is marked (step 1, 
Table 4.3), we can output i. The total order is the sequence of output step 
indexes. 
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Table 4.2. Warshall’s algorithm 



Algorithm Warshall(C) 

Input: A connectivity matrix C. 
Output: A transitive closme matrix C*. 



1. C* := C; 

2. for A: := 1 to n do 

3. for 2 := 1 to n do 

4. for j := 1 to n do 

5. if C*( 2 , j) = 0 then 

6. C*{i,j)'=C*{i,k)^C*{k,3y. 

7. end if 

8. end for 

9. end for 

10. end for 

11. return (C*); 



Table 4.3. A depth-first search algorithm 



Algorithm DEPTHFlRST(i); 

Input: A vertex index i, a connectivity matrix C. 

Output: A depth-first search of vertices starting with vertex i. 



1. Mark(z) := 1; /* i is the vertex being visited */ 

2. let Children be the set of indexes j such that C{i,j) = 1 and 
that there is no such k with C{i, k) = 1, C{k,j) = 1; 

3. for each j € Children do 

4. if Mark(j) = 0 then 

5. DepthFirstQ); 

6. end if 

7. end for 



Similarly, for a directed graph with cycles (not a graph of plan steps), 
if we have a transitive closure C, we can find out all strongly connected 
components by looking for indexes i and j such that C{i,j) = C{j,i) = 1, 
and putting i, j in the same component. 
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Aho et al. [2] present algorithms for depth-first search, topological sort 
and finding strongly connected components based on a linked-list represen- 
tation for graph connectivity. Under this representation, the algorithms can 
be more efficient, although building and updating the list requires additional 
computational overhead. 



4.3 Dynamic Programming 

A frequently encountered problem in planning is finding a sequence of de- 
cisions subject to certain criteria. Each decision represents a choice made 
among a set of alternatives. In plan generation, a decision can be the selec- 
tion of a next action to perform, a choice of a threat resolution method, or a 
way to establish an open precondition. Because the impact of an individual 
decision might not be apparent on the overall cost of a sequence of deci- 
sions, different sequences need be compared in an efficient manner. Dynamic 
programming offers a very useful tool for managing this task. 

The aim of dynamic programming is to find an optimal sequence of deci- 
.sions that leads to a goal, where optimality is defined by the user. Consider a 
set of possible decisions D — {di, c/ 2 , • • • , dn} that an agent might take. Con- 
sider also a space consisting of all possible sequences and subsequences of 
decisions drawn from 'P(D), the power set of D. Each sequence in this space 
leads to a state. From a state 5, one can obtain another state T by choosing 
a next decision d and adding d to 5. Also, suppose that attached to each de- 
cision d(5, T) that leads from 5 to T there is a cost value Cost(d(5, T)). The 
states and their connectivity can be implicitly represented as a state-space 
graph (see Figure 4.2). 

The core of dynamic programming method is the principle of optimal- 
ity. Let 5i, 52, . . . , be the set of all states directly connected to state T 




Fig. 4.2. An illustration of dynamic programming algorithms 
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Table 4.4. The Dynamic algorithm 



Algorithm Dynamic(T) 

Input: A state T. 

Output: A cost value Cost and a set of optimal decisions Opt-Decisions(T), leading 
to state T. 



1. if T is a start state then 

2. return(O); 

3. else 

4. Parent-List := Parents(T); 

5. let 5 be a parent of T in Parent-List such that 

Cost := (Cost(d(5, T)) + Dynamic( 5')) is minimal; 

6. Opt-Decisions(T) := Opt-Decisions(5)U{d(5, T)}; 

7. end if; 

8. return(Cost); 



via a single decision = 1,2, .. .n. The principle of optimality says 

that the optimal sequence of decisions passing through T must be obtained 
from the optimal subsequence of decisions from the initial state Si^it (5'init 
is an empty set of decisions) to T, and that from T to the final state. Let 
MinCost(T) represent the cost of a minimal cost path reaching T from 5init- 
In terms of costs, the principle of optimality translates to the following for- 
mula (see Figure 4.2); 

MinCost(T) = min{ [MinCost(5i) + Cost(d(5i, T))] | i = 1, 2, . . . n} 

The value of this formula is that it is inductive. One can start from an empty 
set of decisions and recursively compute a minimal-cost decision sequence 
incrementally. In each step of computation, the decision d{Si,T) which gives 
rise to a minimal cost value of state T is remembered. When a final state 
is reached, the optimal sequence of decisions can then be traced backwards 
from the goal. 

In Table 4.4 we list an implementation of a dynamic programming algo- 
rithm using recursion. It is assumed that for the problem at hand, there is 
a start state and a final state. The aim is to find an optimal path from the 
start state to the final state via a sequence of optimal decisions. To apply this 
algorithm, one needs a supporting subroutine Parents (5) (see Figure 4.2) for 
generating a set of parent states from any given state. The algorithm can be 
activated by a program call with a final state as an input parameter. 

Dynamic programming is often applied to explicit graphs. A state can be 
formulated by either a single node in a graph or a set of nodes. An advantage 
of dynamic programming is that with the explicit graph as input, the states 
on which computation is made need only be implicitly represented. After 
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each step of computation only the optimal decision is remembered (step 6 in 
Table 4.4), and the rest of the state connections forgotten. Because of this 
feature, dynamic programming can function rather effectively even when the 
set of states Si leading to any particular state T is large. 

In Chapter 8 we consider how to merge plans using an implementation of 
a dynamic programming algorithm. 



4.4 Branch and Bound 

Dynamic programming is a computational method for choosing among se- 
quences of alternative decisions. Another useful alternative-selection method 
is branch-and-bound. Like dynamic programming, this method depends on 
modeling a problem as a state-space search problem. A state can be a config- 
uration of the world much in the same way as the states in the planning 
algorithm ToPlan, or it can be a set of chosen decisions as in the dy- 
namic programming model. It is assumed that given any state 5, a function 
Successors (5) generates the set of all children states from S. These children 
states are then entered in some order into a list of nodes known as an active 
list. In each iteration of a branch and bound algorithm, a state is chosen to 
be expanded, producing its successor states. This state is called an entering 
state. 

The aim of branch and bound is to find a minimum cost goal state. What 
makes branching and bound different from the other algorithms is its ability 
to trim the active list by using two estimated cost values, a lower hound and 
an upper bound cost value for each state. The upper bound U{S) for a state 
5 is a current estimate on how high the cost values of 5 as well as all possible 
descendents of S can reach. If T is a descendent of 5, then Cost(T) < U{S). 
Similarly, a lower bound value L{S) estimates how low the cost values of S and 
the descendents of S can be. For every state 5, L(5) < U{S). Furthermore, 
it is assumed that for a goal state, we know its cost value exactly. If G is a 
goal, then L{G) = U{G) = Cost(G). It is also assumed that for every state, 
these two estimates are provided by the user. 

Suppose that the active list contains two states Si and 52. If L{Si) > 
U{S 2 ) then all descendents of S\ must have a higher cost than any descendent 
of 52- This indicates that S\ can be removed from the active list without 
losing a least-cost solution. This process is shown in Figure 4.3. 

An algorithm for branch and bound is shown in Table 4.5. 

Given the similar goals for both dynamic programming and branch-and- 
bound, how to we choose between these two methods for a given application? 
The “branching” part of a branch and bound algorithm corresponds to the 
generation of children states from a parent state. To make this process feasi- 
ble, branch and bound is usually applied when the number of children states 
is small. In addition, we must have accurate estimates for the upper and lower 
bounds; the tighter the two bounds, the more the number of states pruned. 
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Table 4.5. The Branch-AND-Bound algorithm 



Algorithm Branch-and-Bound(5') 

Input: An initial state S 
Output: An optimal goal state G. 

1. A := {5}; 

/* (A is the branch- and-bound active set)^ j 

2. repeat 

3. remove from A a state S for which L[S) is smallest; 

4. if 5 is a goal state then return(S') 

5. else 

6. C := Successors(5); 

7. for each node S\ in A do 

8. for each node Si in C do 

9. if L(5i) > V{Si) then 

10. remove S\ from A; 

11. else if L{Si) > U{Si) then 

12. remove Si from C; 

13. end if 

14. A:=AUC; 

15. end if 

16. until A = 0; 

17. return(Fail); 



Initial State 




Active List L(S1)>U(S2) 




Fig. 4.3. An illustration of branch and bound algorithms 
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When the number of successors of any state is large, dynamic programming 
is recommended because of its ability to compress all successors generated 
from a given state by an optimal decision state. 



4.5 Constraint Satisfaction 

Constraint satisfaction problems (CSPs) provide a simple and yet powerful 
framework for solving a large variety of AI problems. The technique has been 
successfully applied to machine vision, belief maintenance, scheduling, and 
many design tasks. In Chapters 5 and 7 we will describe how it is used for 
modeling and solving planning problems. 

A constraint satisfaction problem (CSP) can be formulated abstractly as 
three components: 

1. a set of variables^ = 1, 2 . . . n, 

2. for each variable Xi a set of values {vn^Vi 2 ^ . . .Vik}^ Each set is called a 
domain for the corresponding variable, denoted as Domain(ATi), 

3. a collection of constraints that defines the permissible subsets of values 
to variables. 

The goal of a CSP is to find one (or all) assignment of values to the variables 
such that no constraints are violated. Each assignment, {xi = Vij., i = 
1, 2, . . . , n}, is called a solution to the CSP. 

As an example of a CSP, consider a map-coloring problem, where the 
variables are regions Ri,i = 1, 2, . . . , n that are to be colored (see Figure 4.4). 
In a final solution every region must be assigned a color such that no two 
adjacent regions share the same color. A domain for a variable is the set of 
alternative colors that a region can be painted with. For example, a domain 
for i ?3 might be {Green, Red, Blue}. Between every pair of adjacent variables, 
a constraint exists that states that the pair cannot be assigned the same 
color. Between adjacent regions R\ and R 2 , for example, there is a constraint 




Corresponding CSP: 

Variables; R1, R2, R3 

Domains: domain(R1)={Red} 

domain(R2)={Green,Red} 
domain(R3)={Green, Red, Blue} 

Constraints: Ri Rj for i j 



Fig. 4.4. A CSP representation of the graph coloring problem 
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i^i zfz jR 2 , stating that a valucf assigned to Ri must be different from one 
assigned to i? 2 - A solution to the problem is a set of colors, one for each 
region, that satisfies the constraints. 

Let Vars = {X, y, ... Z} be a set of variables. A constraint on Vars is 
essentially a relation on the domains of the variables in Vars. If a constraint 
relates only two variables then it is called a binary constraint A CSP is binary 
if all constraints are binary. For any two variables X and F, we say X = u 
and Y = V are consistent if all constraints between X and Y are satisfied by 
this assignment. 

A variety of techniques have been developed for solving CSPs. They can be 
classified as local consistency-based methods^ global backtrack-based methods^ 
or local-search methods. Local-search methods is a kind of greedy algorithm 
which is gaining popularity. In the last section we review a special case of 
local search algorithms for solving large-scale satisfiability problems. An ap- 
plication of local search to scheduling can be found in a paper by Minton et 
al. [96]. 

4.5.1 Local- Consistency Based Methods 

Local consistency methods follow the theme of preprocessing. That is, before 
a more costly method is used, a consistency-based method can be applied to 
simplify a CSP and to remove any obviously incompatible values. Often these 
methods yield tremendous headway toward eventually solving the problem. 

Let X and Y be two variables. If a domain value A of X is inconsistent 
with all values of F, then A cannot be part of a final solution to the CSP. 
This is because in any final solution 5, any assignment to X piust satisfy all 
constraints in the CSP. Since X = A violates at least one constraint in all 
possible solutions, A can be removed from the domain of X without affecting 
any solution. The procedure is implemented in a subroutine REVISE (see 
Table 4.6). 

Figure 4.5 illustrates the operation of Revise. In this example, the value 
A of a variable X is inconsistent with every value of variable F; A is therefore 
removed from the domain of X without losing any potential solutions. In the 
graph-coloring example of Figure 4.4, this procedure removes the value Red 
from the domain of region R3, making the problem simpler to solve. 

If for a pair of variables (X, F), for every value of X there is a corre- 
sponding consistent value of F, then we say (X, F) is arc-consistent. By the 
above argument, enforcing arc-consistency by removing values from variable 
domains does not affect the final solution. The condition that every pair of 
variables is arc-consistent is called arc- consistency. 

The algorithm AC, described in Table 4.6, enforces arc-consistency for all 
pairs of variables (X, F). It is assumed that for any CSP, Arcs(CSP) gives 
the set of all pairs of variables which are constrained under a binary con- 
straint. The algorithm applies a subroutine REVISE to every constrained pair 
of variables (X, F), to ensure the the pair is arc-consistent. If not. Revise will 
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variable X 



variable Y 




remove inconsistent values from the domain of X. Since the removal of cer- 
tain values from Domain(X) might prompt new violations of arc-consistency 
for all variable pairs (Z,X) G Arcs(CSP), these pairs are added back to the 
queue (Step 6) by AC for re-examination. 

4.5.2 Backtrack-Free Algorithm 

In some special cases, it is even possible to solve a CSP entirely by studying ' 
its graph topology. In this graph every variable is a node, and two nodes are 
connected by an arc if there is a constraint between them. Freuder [53] gave 
the following theorem: 

Theorem 4.5.1 (Freuder 82) 

If (1) the topology of the CSP is a tree, (2) the consistency relations between 
variables are binary, and (3) arc- consistency is satisfied by the graph, then 
the CSP can be solved without backtracking. 

To see this, suppose that a CSP satisfies the three conditions in the above 
theorem. It can be solved by selecting a value from the root node and re- 
peatedly finding a consistent value in a next adjacent node. The procedure 
repeats until a value is selected for every node. Because all relations are binary 
and the graph is a tree, the values in the next adjacent node only interact 
with the values in the immediately previous node. Because arc-consistency 
is satisfied, any partial solution selected is guaranteed to be consistent with 
the next node. Thus, it is possible to extend the current partial solution by 
including one more variable-value pair. By induction, the entire CSP can be 
solved without backtracking. 
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Table 4.6. The AC algorithm 



Algorithm AC(CSP) 

Input: A CSP 

Output: An arc-consistent CSP; 



1. Queue := {{X, Y) \ {X, Y) G Arcs(CSP)}; 

2. loop 

3. select and remove (X, Y) from Queue; 

4. if Revise(X, Y) = TRUE then 

5. affected-pairs := {{Z, X) | (Z, X) G Arcs(CSP)}; 

6. Queue := QueueUaffected-pairs; 

7. end if 

8. until Queue = 0 

9. return(CSP); 



Subroutine Revise(X,F) 

Input: A pair of variables X and Y 

Output: TRUE if the Domain(X) is revised; FALSE otherwise. 



1. Delete := FALSE; 

2. for each u in Domain(X) do 

3. Removeflag:= TRUE; /* if false we know some value is removed */ 

4. for each v in Domain(F) do 

5. if X = u,Y = V are consistent, then 

6. ReMOVEFLAG := FALSE; 

7. end if 

8 end for; 

9. if Removeflag = TRUE then 

10. Domain(X) := Domain(X) — {it}; 

11. Delete := TRUE; 

12. end if 

13. end for; 

14. return(Delete); 



4.5.3 Backtrack-Based Algorithms 

Arc-consistency algorithms only work on pairs of variables, and as such can 
only handle binary constraints and cannot always guarantee a final solution 
to a CSP. A more thorough method for solving a CSP is backtracking, where 
a depth-first search is performed on a search tree formed by the variables in 
the CSP. 

A backtracking algorithm instantiates the variables one at a time in a 
depth-first manner. It backtracks when the constraints accumulated so far 
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Variables: 




signal inconsistency. In Figure 4.6 we show this process. First, variables are 
ordered in a certain sequence. Different orders of variables might entail differ- 
ent search efficiency, and heuristics for good ordering of variables are called 
variable- ordering heuristics. Similarly, for each variable, the values are tried 
out one at a time, and the heuristics for a good ordering of values are called 
value- ordering heuristics. 

We focus on how to combine backtracking and arc-consistency to yield 
a more intelligent backtrack-based algorithm. As shown in the figure, the 
backtracking algorithm searches in a space of nodes. A node consists of a list 
of assignments already made to variables and a list of remaining variables yet 
to be assigned values: 

N — (Assigned, Rem) 

where Assigned = {XI = v\^X2 = U 2 ,-.., Xm = Vm} is the current assign- 
ment, and Rem is a set of variable/domain pairs for all remaining variables: 

Rem = { ^m+1,25 • • •}) 

(-^m+27 {'*^m+2,l7 '^m+2,2? • • •}) 

{^n,l5 ^n,2j ■ ■ •}) 

During the backtracking search, the children nodes of N can be formed 
by assigning one of the values to the first remaining variable Xm+i, 

resulting in a new node: 

Nj = (AssignedU{(A'm+i = Rest (Rem)) 

where the function Rest returns a list with the first element removed. 

For each child Nj^ partial arc-consistency can be enforced on the new 
set of remaining variables: for every remaining variable and its associated 
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Table 4.7. The FwdChecking algorithm 



Algorithm FwdChecking(CSP) 

Input: A CSP: CSP - Domain(Ai)) | i = 

External Functions: 

FwdRevise((X, i;),Reni) updates the variable domains by a new assign- 
ment X = v; 

VarOrdering(CSP) implements a variable-ordering heuristic; 
ValOrdering(Valset) implements a value-ordering heuristic; 
Output: A solution to the CSP if a solution exists; Fail otherwise. 



1. if CSP = 0 then 

return(0); 

2. else 

3. CSP := VarOrdering(CSP); 

4. (X, Valset) := First (CSP); 

5. Valset := ValOrdering (Valset); 

6. for each value v in Valset do 

7: NewA {{Xm+uv)}] 

8. Rem := FwdRevise(NewA, Rest(CSP)); 

9. if (SubSol = FwDCHECKiNG(Rem)) / Fail then 

10. return(NEwAuSuBSOL); 

11. end if 

12. end for 

13. end if 

14. return(Fail); 



domain values (X^, • • •}), remove all values Vi^i for which (X^+i = 

Vj,Xi = Vi^i) is inconsistent. By the same reason as with arc-consistency, 
this value-removal operation does not delete any potential solutions to the 
CSP. If as a result of this processing some value set becomes empty, then the 
entire subtree associated with that child should correspond to a dead end. 
Search need not go further along that direction. This procedure is essentially 
a version of the subroutine Revise. We will call it FwdRevise. 

This simple amendment to backtracking literally checks forward after ev- 
ery node expansion. For this reason it is called forward-checking. Forward- 
checking has been shown to be one of the most efficient methods for solving 
a CSP. Table 4.7 shows the algorithm. 

In the FwdChecking algorithm two heuristic functions are provided by 
the user. The function VarOrdering(CSP) implements a variable-ordering 
heuristic for a CSP, and the function ValOrdering(Valset) implements a 
value-ordering heuristic on a set of values. These heuristics have been tested 
under several schemes, classified under either fail- first or fail-last. The former 
prefers a search-tree branch where it is most likely to terminate without 
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finding a solution, the latter a path that is most likely to succeed. Each of 
these heuristics has been shown to be eflPective in specific domains. 

In addition to forward checking, there are also other intelligent backtrack- 
based algorithms. Some examples are hackjumping and backmarking^ both 
used for implementing ways to record reasons for either failures or success 
during a backtracking search. 



4.6 The Gsat Algorithm 

In Chapter 2 we discussed the SatPlan algorithm for planning. This plan- 
generation algorithm depends on the GSAT algorithm, which we discuss in 
this section. 

The Gsat algorithm is a special type of local-search algorithm, designed 
to solve large-scale propositional satisfiability problems. Its input is a set 
of clauses CSet = {Ci,i = 0, 1, 2, . . . ,n}, representing the conjunction of 
the clauses. Each clause G{ is a set of literals {pi,P 2 ? • • • iPk}, representing, in 
this case, their disjunction. The set of clauses represents a conjunctive normal 
Jorm (CNF). 

In a CNF, each proposition pi can be either TRUE or FALSE. If any propo- 
sition in a clause is set to TRUE, the clause is said to be satisfied. The goal 
of the propositional satisfiability problem is to set at least one proposition pi 
in every clause to TRUE. The resultant solution is known as a satisfying 
assignment. 

Local search can be very effective for several constraint-satisfaction prob- 
lems. Examples of these problems are the A-queens puzzles and the schedul- 
ing problems [126, 96]. In the case of propositional satisfiability problems, it 
is particularly useful. A propositional satisfiability problem is a special kind 
of constraint satisfaction problem, in which the propositions are the vari- 
ables, the true/false values are domain values for the variables, and the 
requirement that all clauses be satisfied corresponds to a global constraint 
on the variables. 

Given a set of clauses CSet, Gsat starts by generating an initial assign- 
ment T. This assignment may fail to set all clauses true. In that case, Gsat 
finds a proposition such that reversing its truth value (from TRUE to FALSE or 
vice versa) results in the largest increase in the number of satisfied clauses^ . 
This proposition is called a max-satisfying proposition, and the operation for 
changing the truth value of p is known as a flip. After each flipping operation, 
a next assignment is generated by Gsat by reversing the truth value of this 
proposition. GsAT repeats this process until either a satisfying assignment is 
found, or a pre-assigned upper bound on the number of flips is exceeded. 



^ These are occasional exceptions. For example, it might only require the smallest 
decrease in the number of failed clauses. 
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Table 4.8. The Gsat algorithm 



Algorithm GsAT(CSet); 

Input: A set clauses CSet, two constants MaxTries and MaxFhps. 
Output: a satisfying truth assignment of CSet if found. 



1 for z := 1 to MaxTries do 

2 T := a randomly generated truth assignment; 

3 for j := 1 to MaxFhps do 

4 if T is a satisfying assignment then 

5 return T ; 

6 else 

7 let p be a max-satisfying proposition 
(break ties randomly); 

Comment: now flip p... 

8 ifp := TRUE in T then 

9 set p := FALSE in T; 

10 else 

11 set p TRUE in T; 

12 end if 

13 end if (step 4) 

14 end for (step 3) 

15 end for (step 1) 

16 return(Fail); 



For example, suppose that a CNF CSet consists of two clauses: T = 
{Cl, C 2 }, where the clauses are 

Ci= {x,y,-^z), (4.1) 

C 2 = {-'X, -■?/, -iz} (4.2) 

Assume that an initial assignment T is defined as x = TRUE, y = true, z = 
TRUE. Under this assignment, C 2 is false. In this case, z is a max-satisfying 
proposition since setting it to FALSE makes both clauses true. In fact, setting 
z to FALSE gives rise to a satisfying assignment. 

A high-level version of the Gsat algorithm is shown in Table 4.8. Two 
parameters must be determined before the application of the algorithm. One 
is MaxTries, designating the maximum number of local-search sessions to try 
before giving up. The second is MaxFlips which gives an upper bound on the 
number of flips for satisfying propositions during each local-search session. 
By default, MaxTries and MaxFlips can be set at N and 5iV, respectively, 
for a problem with N propositions. 

A feature of Gsat is that when choosing which proposition to flip, it is 
sometimes beneficial to allow a flip to occur even when it does not strictly 
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increase the number of satisfied clauses. Also, a certain amount of non- 
determinism is found to be useful. For example, if there is more than one 
proposition whose flipping would result in a maximum increase in satisfied 
clauses, one can select among them at random. In addition, a few variations 
of the basic GSAT algorithm are found to be very effective. The most impor- 
tant variation on GSAT is known as the “walk” option, where there is a new 
probability parameter P whose value lies between 0 and 1. On each flip, with 
probability P, any variable that appears in a unsatisfied clause can be flipped. 
Likewise, with probability (1-P), a max-satisfying variable is flipped. This 
option is necessary for solving the planning problems discussed in Chapter 2. 
As an example of the variations of Gsat, see [120]. 

For several test problems, Gsat dramatically outperforms backtracking- 
based search algorithms, often resulting in orders of magnitude reduction 
in search time. There has been much study on what constitutes the hard 
problems for GSAT. It was discovered that some problems with a certain 
proposition/clause ratio are the hardest to solve using Gsat as well as other 
CSP algorithms. See [34] for a discussion. 



4.7 Background 

Unification algorithms can be found in several AI textbooks, including those 
by Nilsson [102], Russell and Norvig [112], and Rich and Knight [110]. Graph- 
traversal algorithms are thoroughly discussed in a book by Aho, Hopcroft, 
and Ullman [2]. Dynamic programming and branch-and-bound methods can 
be found in [66]. 

Since the early 1980s, Constraint Satisfaction has undergone a tremendous 
growth both in interest and in application. Early foundational work can be 
found in [92], [91], [53], [37], [36]. Algorithm AC is based on [92]. Forward- 
checking is described in [62]. Good overviews on this topic can be found in 
an article by Kumar [84] and a book by Tsang [162]; a collection of papers 
can be found in a special issue of Artificial Intelligence Journal on constraint- 
directed reasoning [54]. 

The Gsat algorithm is based on an IJCAI-93 article by Selman and Kautz 
[118]. There is a large body of work related to local search techniques and 
propositional satisfaction problems. See [30], [120, 34], [118], [119] for an 
overview. See [77] for a discussion on how Gsat is applied to planning. 
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As a case study of the application of constraint-satisfaction representation, 
in this chapter we consider how to extend a traditional backward-chaining, 
partial-order planner to one that could reason about resources. We present 
an extended-plan language in which each variable is associated with a finite 
set of domain values. We discuss the associated changes this brings to plan- 
ning algorithms. In addition, we present empirical results to demonstrate the 
efficiency gain of the extended planner, and to answer some of the utility 
questions regarding planning with constraint satisfaction. 



5.1 Eager Variable-Binding Commitments 

Over the years, researchers have extensively studied different methods of 
imposing ordering constraints on the steps of a plan. It has been shown 
that, when appropriately used, a delayed- commitment or least- commitment 
strategy in step-ordering could effectively reduce a planner’s search space. 

In this chapter, we extend the delayed-commitment strategy in step- 
ordering to variable binding^ using techniques in solving constraint-satisfac- 
tion problems (CSPs). The idea is to represent possible bindings of a variable 
in a plan collectively, and to instantiate a variable to a constant value only 
when necessary. This approach to reasoning about resources has been adopted 
by a number of practical planning systems, most notably MOLGEN [128], 
SiPE [147], Gemplan [88], and scheduling systems such as Isis [50]. What 
has not been clear from these previous systems is how to represent the re- 
sources in a concise mathematical model of CSP, and how the associated 
constraint reasoning systems can cope with incremental changes in a plan, 
which are brought up by the insertion of new steps. In addition, there is an 
open utility question on how much constraint processing should be performed 
during planning to best improve the planning efficiency. 

To illustrate the variable-binding problem, consider the household do- 
main. Suppose that our goal is to paint a ceiling (see Figure 5.1). One of 
the last steps in a solution plan is apply-paint, shown at the top of the 
figure, which has an argument ?brush. When this step is inserted in the 
plan, ?brush is yet uncommitted to any particular constant. At this point, 
we have two choices: either we arbitrarily choose an available constant for 
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Goal 



Step 




Preconditions: 
Brush(?brush), ... 






Step 



dip-brush-in-tray(?brush, Tray) 

Preconditions: 
Fit(?brush, Tray), ... 



Initial State 



Brush(B1), Brush(B2), ..., Brush(BIOO), Fit(B50, Tray), ... 

V J 



Fig. 5.1, Committing to a paint brush 



a brush, and instantiate ?brush to that constant, or we wait until we have 
more information before committing to a specific brush. 

The first choice could lead to serious computational problems. As shown 
in Figure 5.2, if there were 100 different brushes, the search tree would split 
into 100 branches. If, however, another step dip-brush- in-tray, for dipping 
the brush in a paint tray, is inserted into the plan at a later stage, it may be 
discovered that most of the 100 brushes would not fit into the tray! At this 
point, many of the branches will correspond to dead ends in the search tree, 
and at these branches extensive backtracking would occur. 

This example exposes a problem with the ad-hoc method of eager commit- 
ment in variable binding. As shown above, the consequence of eager commit- 
ment could make the search tree very bushy, and could lead to a backtracking 
phenomenon known as thrashing. Thrashing occurs in a search process when 
the same part of the search tree fails repeatedly for the same reason, in this 
example the reason being that the brushes would not fit into the paint tray. 
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Dead end Dead end 

Fig. 5.2. The thrashing problem in search 



If this is discovered late in the planning process, a significant amount of 
computation would be wasted. 

A solution to the above problem would be to represent collectively the 
objects to which each variable can be bound. In the next section, we describe 
a representational method for dealing with this problem. 



5.2 Extending the Representation 

In our extension of planning domain representation, every variable has a 
domain of values^ representing the entire range of values that could be as- 
signed to the variable. For example, in the household domain, a painting 
operator apply-paint(?brush. Ceiling) would be associated with a domain 
for ?brush: {^1, B2, . . . , 5100}. For convenience, we could declare a symbol 
BrushDom to be this set, and use the representation in Table 5.1 to represent 
the operator. The denotation of a variable in this extended operator repre- 
sentation is existential] for each variable ?x in a literal P(?x), the associated 
variable declaration 

Var: ?x ; VarDomain 

stands for 

3 ?x £ VarDomain. P{?x) 

In addition, in the initial state representation, all literals sharing the same 
property are represented succinctly using a set notation. For example, in the 
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Table 5.1. Operator definition with resources 



Define BrushDom: {Bl, B2, . . . , BlOO} 

OPERATOR: apply-paint(?brush, Ceiling) 

Var: ?brush: BrushDom; 

Precondition Effect Cost 

Holding(?brush), Painted(Ceihng), 5.0 

Dipped(?brush, Tray), -•Dry(?brush) 

Can- Reach ( Ceiling) 



painting domain, part of the initial state could be represented as follows: 

Define BRUSHES: {Bl, B2, . . . , BlOO}; 

Define SMALLBRUSHES: {B50,B100}; 

Brush(BRUSHES); 

Fit(SMALLBRUSHES, Tray) 

etc. 

In the representation of the initial state, the Define part declares a new 
symbol, which represents a set of objects. The meaning of each literal is 
universal; the literal holds for each of the constants in the set. 

A plan U is represented just as before, except that now each step is 
associated with a set of variable-domain declarations, just as in an operator 
definition. The variables in a plan form a constraint satisfaction problem 
(CSP). In this problem, the set of variables in the steps of a plan are the 
variables of the CSP, the constraints among variables are either equality or 
inequality constraints. A solution to the problem is an assignment of values 
taken from the domains of the variables to the corresponding variables, such 
that all variable-binding constraints (that is, ?xi = 1 x 2 , and lyi ^ly 2 ) are 
satisfied. 

In general, solving this CSP may not be computationally easy. As an ex- 
ample, Figure 5.3 shows a CSP associated with a painting plan. The variables 
are brushes to be used to paint the objects. Clearly there is no solution to 
this CSP, but the computation is nontrivial; in general, constraint-satisfaction 
problems are NP-hard. 

Despite the worst-case computational difficulty in solving a CSP, efficient 
algorithms exist. As mentioned in Chapter 4, the local consistency algorithms 
and global backtracking-based heuristic algorithms could help detect dead 
ends early and solve a CSP efficiently. In this chapter, we refer to these CSP 
algorithms collectively as CspSOLVE. Given a constraint satisfaction problem 
CSP, CspSolve(CSP) returns a solution to the problem, where a solution is 
a consistent assignment to all variables. If the CSP has no solution, CspSOLVE 
returns FALSE. 
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Preconditions: Effects: 



Goal 



dry(?b1) 



dry(?b2) 



dry(?b3) 




CSP: 



?b1 ?b2 

?b3 It ?b2 
?b1 ?b3 



?b1:{B1,B2} 
?b2: {B1, B2} 
?b3: {B1, B2} 



Fig. 5.3. A constraint satisfaction problem associated with a plan 



5.3 Plan Generation 

Consider a backward-chaining, partial-order planning algorithm. To cope 
with the new representation, we must extend several key steps to account for 
constraint reasoning. Readers should consult the backward-chaining, partial- 
order planning algorithm in Table 2.2. The resultant extended algorithm will 
be referred to as FdPop algorithm (Finite-Domain Partial-Order Planner). 

5.3.1 Correctness Check 

The correctness-checking step (Step 5) of a partial-order planning algorithm 
verifies the plan to make sure that every precondition of every step has a 
causal link, and that the causal link has no negative threats. To extend to 
the CSP representation, all we need add is that in addition to the above 
check. Step 5 will also verify if a consistent solution to the associated CSP 
exists. This can always be done using a backtracking algorithm. If so then a 
correct and consistent instance of the plan Instance(77) will be returned. 

5.3.2 Threat Detection 

Threat detection is needed in Poplan at step 7. The modification for 

constraint-based threat-detection is simple: for each causal link (s^ s^), a 

check is made to see if a step Sk exists such that 
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a. Sfc could be ordered between and s^ , 

b. Sk has an effect -^p{y), such that x and y are not constrainted by 
a separation constraint, and 

c. if X and y are variables, Domain(x) fl Domain(y) ^ 0. 

d. If both X and y are constants, x = y. 

e. Otherwise, if one of x and y is a constant C and the other a 
variable, C is in the domain of the variable. 

Item c above indicates that Sk is a possible threatening step. 

5.3.3 Precondition Establishment 

The next extension to the planning algorithm is in the 
Establish“Pre condition step (Step 10). Let Sj be a plan step with an open 
precondition p(?x), where the domain of the variable ?x is Domain(?x). This 
precondition can be achieved by an operator in <9, or by an existing step. Of 
all existing steps, one of them could be the start step, representing the initial 
state. Here we consider the start step and the rest of the steps separately. 

Case 1: Start Step. Suppose that the start step asserts an effect literal p(Set), 
and that Set and Domain (?x) have a non-empty intersection. We can let 
Domain(?x) be this intersection, and we can produce a new causal link 

(Sinit Sj) in the plan. 

Case 2: Other Steps and New Operators. Let be an existing step with an 
effect literal p(?y). If Si could be ordered before Sj without violating ordering 

constraints, we would like to make (s^ s^) a new causal link in the plan. 

In doing so, we would like to make sure that ?x and ?y have a non-empty 
domain intersection. If this is true, we let the domains of both ?x and 7y be 
the same as their intersection. The procedure for inserting a new operator to 
achieve a precondition requires a similar modification. 

The FdPop algorithm is shown in Table 5.2. 



5.4 A House-Painting Example 

We now illustrate the FdPop algorithm with the painting example intro- 
duced in Chapter 2 (Section 2.3.4), where our goals are to paint a ceiling, 
a door and a table. To achieve these goals using the new operator defini- 
tions, suppose also that we re-define the operators in Table 2.3, resulting in 
Table 5.3. Each painting operator has a list of possible brushes that could 
be used for painting an object. The brushes are collectively represented as 
variables. 

The first several planning operations are basically identical to those de- 
scribed in Section 2.3.4. We focus on the last step, shown in Figure 5.4. In 
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Table 5.2. The FdPop algorithm 



Algorithm FdPop((!?, 

Input: A set of planning operators O, and an initial plan consisting of a 

start step and a finish step; 

Output: A correct plan if a solution can be found. 

1 OpenList := { iljjjjt}. 

2 repeat 

3 77:= lowest cost plan in OpenList; 

4 remove 77 from OpenList; 

5 if Correct(77) then 

6 if CspSolve( 77) then 

7 return(Instance(77)); 

8 end if 

9 else 

10 if Threats-Exist(77) then 

11 Let t be a threat; Succ := Resolve-Threat (t, 77); 

12 else 

13 Succ := Establish-Precondition(77); 

14 end if 

15 Decide whether to apply CspSolve to all successors in Succ; 

If yes, then apply; 

16 Add all remaining successors in Succ to OpenList; 

17 end if 

18 until OpenList is empty; 

19 return(Fail); 



Table 5.3. Operator definition with resources 



paint-ceiling(?cbr, Ceiling) 

Var: ?cbr: {B1,B2,B3} 

Preconditions Effects Cost 

Dry(?cbr) Painted (Ceiling), 5.0 

-iDry(?cbr) 

paint-door(?dbr. Door) 

Var: ?dbr: {B1,B2} 

Preconditions Effects Cost 

Dry(?dbr) Painted (Door), 5.0 

-iDry(?dbr) 

paint-table(?tbr. Table) 

Var: ?tbr: {Bl} 

Preconditions Effects Cost 

Dry(?tbr) Painted (Table), 5.0 

-'Dry(?tbr) 
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?cbr ?dbr 
?cbr ^ ?tbr 
?tbr^?dbr 



?cbr: {B1,B2,B3} 
?dbr: {B1,B2} 
?tbr: {B1} 



converted to 
aCSP 



1 ?cbr: {B1,B2,B3} 1-^ 

V ^ J ^ 

I "J ?dbr; {B1.B2} 

^ y 

I ?tbr: {B1} r ' ^ 



Fig. 5.4. A finite-domain plan where the variables form a constraint satisfaction 
problem 




?cbr ?dbr 
?cbr ^ ?tbr 
?tbr^?dbr 



?cbr: {B3} 
?dbr: {B2} 
?tbr: {B1} 



Fig. 5.5. A final solution plan for the house-painting example 
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this plan every precondition has an associated causal link, and Step 5 of Fd- 
POP algorithm is encountered. At this point, a CSP solver processes a CSP 
in Figure 5.4. When examining a variable pair, (?tbr, ?dbr), arc-consistency 
found that the value B1 can be removed from the domain of the latter. 
Similarly the value B1 is removed from the domain of ?cbr. When the pair 
(?dbr, ?cbr) is processed, the value B2 is removed from the domain of ?cbr. 
The resultant singleton domains constitute a final solution to the CSP. And 
the corresponding solution plan is shown in Figure 5.5. 

This example demonstrates several advantages of FdPop. Suppose that 
during planning, we introduce a plan step with a variable ?x, where ?x can 
be bound to n different constants, al,a2, . . . ,an. If we next insert another 
step with a parameter ly that can be instantiated to any of m constants 
61, 62, . . . bm, then the state space of a traditional partial-order planner might 
look like Figure 5.6a. In this search tree, the partial-order planner commits 
to one of al through an at one level, and then commits to one of 61 through 
6m at another level. In the worst case, there are tight constraints on their 
values which can only be satisfied by constants an and bm, and the entire 
search tree with a size of m * n must be scanned. 

In contrast to the traditional approach, the extended constraint-based 
planner does not commit to any specific constant during planning. The do- 
main values for both ?x and ly are simply recorded and reasoned about at a 
later time. Therefore the search tree looks like the one shown in Figure 5.6b. 

o 



dom(?x) = a1 , ... an 



dom(?y) = b1 , ... bm 



b 




a 

Fig. 5.6. Comparing TopPop and FdPop 
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5.5 A Variable-Sized Blocks World Domain 

In the above we have shown a representational method for reasoning about fi- 
nite resources available to planning operators. An important remaining ques- 
tion is how often constraint satisfaction routines should be applied during 
planning. We have several options in implementing the constraint-checking 
part of step 15 in FdPop. Here we could choose to verify whether a solution 
exists to the CSP in every plan expansion, every other expansion, or every 

expansion. It is a well-known utility question to determine which option 
works best. 

In this section, we do not attempt to address the utility question fully, 
but instead present an illustrative example and show how the utility issue for 
other planning domains could be similarly investigated. 







^2 


• • • 


Bn 


A 



Fig. 5.7. The variable-sized blocks domain 



The domain we look at is a variable-sized blocks world domain. In this 
domain there are a number of blocks J5i, H 2 , . . . , of unit size, and there is 
a particular block A that can support more than one other block. The task 
is to move all the B blocks on top of block A, using the operator shown in 
Table 5.4. 

In this definition, represents a location available on block A. The vari- 
able ?s could be bound to any location on A. By varying m, we can change 
where a dead end occurs in the search tree, as well as the percentage of cor- 
rect and consistent solutions among the leaf nodes of the tree. We call the 
latter the density of solutions. 

In this domain we ran an empirical test, where the number of B blocks 
is fixed at five, and the number of locations on top of block A varies from 
two to five. A dead end occurs very high up in the search tree if the number 
of locations is two; it occurs relatively deep in a search tree if the number of 

Table 5.4. Operator definition with resources 



PutonA(?a;, ?s) 

Var: lx : {Bi, B 2 , . . • , Bn}; Is : {Li, L 2 , . . . , Lm}; 

Precondition Effect Cost 

Clear(?x), SpaceOnA(?s) OnA(?x), -iSpaceOnA(?s) 1 
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Fig. 5.8. Performance in variable-sized blocks domain 



locations is four. When the number of locations is five, all five blocks can be 
put on block A at the same time. In this case no dead end occurs. 

Our implementation of FdPop is an extended Snlp algorithm [17], which 
is run on a SUN4/Sparc Station using Allegro Common Lisp. We call the 
resulant planner FSNLP. 

Figure 5.8 depicts the empirical results, where the frequency of CSP ap- 
plications refers to the inverse of the number of nodes, minus one, along a 
search path between any two successive CSP applications: the frequency is 
one if CspSolve is applied to every node, two if every other node. If Csp- 
SOLVE is not applied to any node at all, then the frequency is 0. The figure 
shows that when the dead end occurs early in the search tree, for example, 
when the number of locations is 2 (the solid line in the figure), the CPU time 
consumed is in inverse proportion to the frequency of CSP application. In the 
opposite case, when the number of locations is many (five), the situation is 
reversed: the CPU time used is in direct proportion to the frequency of solv- 
ing the constraint network, implying that it is better to apply CSP checking 
infrequently. 

The result makes sense intuitively. If the number of locations on block A 
is few, then the dead end occurs high up in the search tree, and therefore 
when CspSolve is applied frequently these dead ends are detected as soon 
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as possible. In this case, frequent application of CspSolve could save a lot 
of effort. In the opposite case, most of the tree paths in the search tree 
lead to solutions, and therefore applying CspSolve wiU not provide much 
computational advantage. In this case, it is better to apply CSP solving 
infrequently. 

Lastly, it should be emphasized that the utility question is still very much 
an open question. Although we have shown that in this restricted domain it 
is possible to understand the impact of the problem fairly well, in general 
much more research is still needed. 



5.6 Summary 

As a case study in applying constraint satisfaction to planning, we have de- 
scribed how to extend a basic planning algorithm, PoPLAN, to include the 
capability for reasoning about finite resources. We can see that such exten- 
sion, in fact, requires only minor changes to the algorithm, but the poten- 
tial efficiency return is far greater. We argued for collective representation 
of constant objects in a planning domain, and proposed to use well-known 
constraint reasoning techniques for handling binding constraints. Finally, we 
provided empirical results in a variable-sized blocks world domain, demon- 
strating how the utility issue regarding CSP applications could be investi- 
gated. 

In subsequent chapters, we will continue to use the basic planning domain 
representation, introduced in Chapter 2. This is mainly due to its simplic- 
ity. We should bear in mind, however, that these subsequent techniques in 
resource reasoning could in fact be extended in a conceptually simple manner. 



5.7 Background 

Reasoning about constraints is a major aspect of knowledge-based schedul- 
ing; a comprehensive coverage can be found in [50], [51] and in a collection 
of articles [52]. Stefik demonstrates that it is more advantageous to com- 
bine planning and resource reasoning [128]. The MOLGEN system functioned 
mainly in the domain of biological and molecular experiments, and as such 
it had certain limitations. Stefik’s experiments, for example, demonstrated 
that a certain partial arc-consistency processing in reasoning about variable- 
binding constraints could lead to efficiency gain, but did not address resource 
reasoning in the context of a formal model of constraint satisfaction. 

Constraint-directed reasoning is becoming a necessary component of real- 
world AI planning systems. Examples of successful combination of the tech- 
niques include Wilkins’ SiPE [147], Lansky’s Gemplan [88], and Ghallab and 
Laruelle’s IxTeT [56]. Constraint reasoning in temporal reasoning has also 
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been one of the research themes of Allen [7, 3]. Joslin and Pollack proposed 
a method for representing and reasoning about all planning decisions as con- 
straints [69]. Finally, Kautz and Selman proposed a method for encoding ac- 
tions as state axioms, and use them in a GsAT-based planning algorithm [77]. 

The work reported in this chapter is adapted from an AIPS94 paper by 
Yang and Chan [155]. More experimental results can be found in Chan’s 
Master’s thesis [26]. 




Part II 



Problem Decomposition 
and Solution Combination 




Overview 



Profound and focused 
and we will be strengthened, 

Confused and divided 

and the opponents are weakened. 

Focused and we act as one 
Weakened as they divide in ten. 

(Sun Tzu, The Art of Strategy, Chapter Six) 



In this part we provide a computational framework for practising divide- 
and- conquer. We first discuss how to decompose a domain automatically 
in Chapter 6, based on an analysis of the inherent interactions among the 
goals. The result of the decomposition process would be a partition of a 
set of goals into subsets, such that the goals in each subset could be solved 
concurrently. When they are combined, the solutions to the subsets might 
still have conflicts between each other. Thus, in Chapter 7 we develop a 
constraint-based method for resolving the conflicts efficiently. Moreover, in 
Chapter 8 we discuss how to take advantage of the overlapping capabilities 
between solution plans by merging them. Finally, Chapter 9 deals with the 
problem of how to select solution plans from among a pool of candidates. 




6. Planning by Decomposition 



The ability to decompose a complex problem into manageable sub-compon- 
ents is a necessity for many intelligent problem-solving activities. Presented 
with a complex planning problem, an intelligent agent following the divide- 
and- conquer methodology tries to decompose it into subproblems, so that 
each can be solved separately. The solutions are then reassembled for an 
overall solution. When performing the decomposition, careful attention is 
paid to the partitioning process so that clean interfaces with controlled in- 
teractions remain. A very important issue is to contain the complexity, and 
limit the number and variety of mechanisms needed to solve each subprob- 
lem. In this chapter we outline an approach to planning using decompo- 
sition and discuss its differences and similarities with other planning algo- 
rithms. 



6.1 Decomposition Planning 

6.1.1 Two Examples 

A Household Example. We begin by considering the following household ex- 
ample. Suppose that on a weekend, we plan to repaint our house and do some 
grocery shopping. Our list of subgoals are the following: 

Filled(Ceiling-Hole), Painted(Ceiling), Painted (Window), 

Painted (Wall), Painted (Door), Have(Bread), Have(Fruit), 

Have( Vegetables), Have(Milk), Have (Detergent), ... 

To work on the ceiling and to complete the painting jobs, we need to 
wear a pair of gloves, to change to working clothes, to get a ladder, and to 
get the painting brushes and paint. To do grocery shopping we need to wear 
somewhat better clothes, to drive a car to the supermarket, and to get each 
item in turn. In the end, we have to pay for all these items. 

A sensible agent is likely to partition these tasks into two classes, one for 
house painting and the other for shopping. When planning for this household 
problem, a plan can be separately developed for achieving the goals in each 
class, and, in the end, the two plans can be combined. 
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A Multi-table Blocks World Example. Next, we consider a scenario where 
there are 10 tables, Tables, z = 1,2,..., 10. On each table i is a stack of 
blocks, Ai,Bi^ etc. Our goal is to arrange the stacks on the tables from their 
initial configurations to their goal configurations. We have one restriction, 
though, requiring that no block can leave its own table. 

In this domain, our intuition tells us that it is better to develop plans 
for transforming block-configurations on each table, in a concurrent manner. 
This decision is made despite the contention among the ten tasks for the 
robot hand as a resource. 

6.1.2 Global Planning-Domain Decomposition 

In both problems above, we decided to decompose a large set of goals into 
several subsets and work on each subset separately from the rest. The de- 
composed goals entail a partition of the planning domain into disjoint sub- 
sets, where each subset has its own operators and initial state. A global- 
decomposition planner develops a plan for each subset of goals concurrently 
with others. A subsequent step resolves conflicts among the separately devel- 
oped plans and merges these plans. 

More precisely, we advocate a global-decomposition planning strategy. 
Let a planning problem be {init, goal^ O) , where init is a set of initial state 
literals, goal are the goal literals, and O is the set of domain operators. As- 
sume that the problem is decomposed into m parts, such that for each part 
{initio goal^^Oi) ^ the initial state mitj, goal state goal^^ and operators Oi are 
subsets of init, goal, and O, respectively. Furthermore, assume that the union 
of initial state initi, goal state goal^, and operators Oi, ^OT i = 1,2 .. .m, give 
rise to the original sets init, goal, and O, respectively. The algorithm for 
global-decomposition planning is shown in Table 6.1. 

There are five critical components in this algorithm. The first compo- 
nent is a method for decomposing a domain into several subparts. This is 
done by the algorithm Gdecomp. We will present an analysis of the bene- 
fits of decomposition in the next section. Through this analysis we provide 
some guidelines on good domain decomposition which can lead to improved 
planning efficiency. The second component in this algorithm is a method for 
finding solutions for each decomposed sub-domain. To implement this com- 
ponent, we could employ any of the basic algorithms discussed in Chapter 2; 
a criterion on which algorithm to select is also discussed in the next section. 
The last three components are for selecting a solution for each sub-domain, 
resolving conflicts among the sub-solutions and merging plans by removing 
redundant parts in the combined solutions. These three components are dis- 
cussed in detail in Chapters 9, 7, and 8, respectively. 
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Table 6.1. The DecomPlan algorithm 



Input: A planning domain and a planning problem {ini% goal). 

Output: A solution plan II for solving the planning problem. 

Algorithm DecomPlan((9, inil goal) 

1. apply the decompositon algorithm Gdecomp() , 

obtaining m subproblems {prob^ = {initi, goal^, Oi), i = 1,2,... m}; 

2. For each problem prob^, find a set of solutions {IIij,j = 1,2,..., fe}; 

3. Select one solution plan Uij for each subproblem; 
let the selected solutions be Solution; 

4. Resolve conflicts among the plans in Solution; 

5. Merge the plans in Solution; 

6. If steps 3 and 4 are successful, return the resultant solution plan. 

7. If every possible combination of solutions has been considered, exit 
with failure. Otherwise, go to step 2. 



6.2 Efficiency Benefits of Decomposition 

One of the early analyses was provided by Korf [83], who discussed com- 
putational complexity issues related to decomposing a compound goal into 
subgoals. His analysis is based on several strong assumptions, and is implic- 
itly based on forward-chaining, total-order planners. Below, we review the 
key points in his analysis and present our extensions. In the subsequent anal- 
ysis, we use No~Decomp to denote a generic planner which solves a conjunctive 
goal without using decomposition, and Use-Decomp for a planner which uses 
decomposition. 

6.2.1 Forward- Chaining, Total-Order Planners 

Korf’s Analysis. First, suppose that to solve each subgoal gi separately, 
the branching factor of a forward-chaining, total-order planner is Fi. That is, 
in the process of achieving gi^ for any state there are Fi applicable operators. 
Suppose also that the optimal solution length for a subgoal gi is Ni. Then 
when all the goals are solved together using a forward-chaining planner, at 
each state all applicable operators must be used to generate successors. When 
all subgoals are completely independent, the worst-case time complexity can 
be derived from equation (3.2) in Section 3.2: 

m 

Time(No-Decomp) = 0((^^ * T^) (d.l) 

i=l 

- ( 6 . 2 ) 

In the above formula, F is the maximum of for i = 1, 2, . . . ,m. The Tj 
factor denotes the constant time spent in each node expansion. 
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Now suppose that, as Korf proposed, we decompose the goals into m 
subgoals, solve each subgoal in turn, and then combine all solutions by con- 
catenation. Since all m goals are completely independent of each other, the 
worst-case time complexity is 

m 

Time(Use-Decomp) = (6.3) 

i=l 

= 0{m*F^) (6.4) 

where N is the maximal optimal solution length for the subgoals. 

Based on the above formulas, Korf’s conclusion is therefore that in the 
case of complete independence, and when forward- chaining planners are used, 
decomposition cuts down both the branching factor and the depth of search, 
leading to an exponential amount of savings in planning time. 

Interacting Goals. We now extend Korf’s analysis to interacting subgoals. 
Now, instead of requiring that the operators for each goal Qi be completely 
independent of the operators for any other goal gj, suppose that we allow a 
certain number of interactions. In particular, we allow 

1. an operator relevant for achieving one goal to delete the precondition of 
an operator relevant for another goal. This is called a deleted- condition 
interaction. Deleted-condition interactions are caused by the presence of 
negative threats in the plans. They are also called conflicts. 

2. an operator relevant for achieving one goal to add the precondition of 
another operator. This is called an operator- overlapping interaction. 

As an example, consider two plans for painting the ceiling and painting the 
ladder, shown in Figure 6.1. The operators get -ladder, one from each plan, 
can be combined into a single one. This is an example of the operator- 
overlapping interactions. The paint-ladder operator interacts with the 
paint-ceiling operator through a deleted-condition interaction; painting 
the ladder might leave the ladder with wet paint, making it unusable for 
painting the ceiling later. 



Goals: 




Fig. 6.1. Interactions in the household domain 
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When these two interactions are both present, the analysis becomes more 
complex. We now not only have to consider the cost for solving the individual 
goals, but also the cost for combining the solutions to form a global solution. 

A. No Decomposition. We analyze the search behavior of a forward-chaining, 
total-order planner when all goals are solved together. Due to the operator- 
overlapping interactions, some operators relevant for achieving one goal may 
themselves be part of operators relevant for achieving another goal. Thus, 
when all goals are solved together, the branching factor of search is somewhat 
smaller than the sum of all individual branching factors Fi. 

As for the search depth, both the operator-overlapping interactions and 
the deleted-condition interactions will contribute to it in various ways. 

First, due to operator-overlapping interactions, some operators may serve 
to establish more than one subgoal. Therefore, the effect of operator-overlap- 
ping interactions is that a certain number of operators No can be subtracted 
from the sum when computing the total solution length. 

Second, due to deleted-condition interactions, some operators for achiev- 
ing one subgoal in a solution may delete a precondition of another operator 
for achieving a different subgoal. To patch up the deleted preconditions, extra 
operators may have to be inserted. Therefore, the effect of deleted-condition 
interactions on total solution lengths is that a number Nd of operators have 
to be added to the sum when computing the total solution length. 

Putting the above factors together, when solving all subgoals at the same 
time, the worst-time search complexity is 

Time(No-Decomp) = O Fi - j (6.5) 

B. Decomposition. With decomposition, the planning process is completed 
in three stages. 

Stage 1. The set of all goals is decomposed into multiple subsets gi,i = 
Stage 2. For each goal set gi, a solution is found; 

Stage 3. The solutions to the m goal sets are combined into a global solution. 

It is also possible that at stage 3 we meet a dead end and therefore the 
solutions cannot be combined into one that achieves all goals. In that case, 
alternative solutions must be planned for in stage 2. 

We assume that the computational complexity for performing the decom- 
position is Time(GDECOMP), where Gdecomp is a goal-directed decomposi- 
tion algorithm. The complexity for the second stage is the same as the case 
when there is complete independence; to solve each subgoal using a forward- 
chaining, total-order planner, the time complexity is 0{F^^). Here we con- 
centrate on the complexity for the third stage, the solution- combination stage. 

Let Id be the total number of deleted-condition interactions among the 
actions for achieving the goals. To resolve these interactions the worst-case 
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time complexity is where r is the number of ways to resolve each 

conflict. We assume that r > 1. 

Related to the deleted-condition interactions, there are also Iq operator- 
overlapping interactions. For each possible operator overlap we have to decide 
whether to “merge” the two operators into one. For example, in the household 
domain, it is possible to get a ladder only once while satisfying a precondition 
for both painting the ceiling and painting the wall. In the worst case, the time 
complexity for taking care of all operator-overlapping interactions is 0(2'^°), 
which is also exponential in Iq. 

Between deleted-condition interactions and operator-overlapping interac- 
tions, there is also a certain amount of interaction. One way of resolving a 
deleted-condition interaction may turn out to make a merging operation im- 
possible. Therefore, when both types of interaction are present, the worst-case 
time complexity for resolving both is 

Finally, in stage 3 we may fail to find a consistent solution to all goals. In 
that case, a certain amount of backtracking to stage 2 must occur. Let k be 
the number of plans for a goal gi. A total combinations might have to be 
considered in the worst case before finding a consistent global solution. Thus, 
the time complexity for solving the planning problem with decomposition, 
using a forward- chaining, total- order planner, is 

Time(Use~Decomp) = 0(Time(GDECOMP) + 

(m * + r^Id+Io)) * yfcm) (6.6) 

C. Summary: Interacting Goals. When the goals are interacting, and when 
a forward-chaining, total-order planner is used, decomposition no longer has 
definite advantages. For decomposition to win, the number of operator-over- 
lapping interactions and deleted-condition interactions must be small. Low 
operator-overlapping interactions imply a small lo value in equation (6.6) 
and small No in equation (6.5). Similarly, low deleted-condition interactions 
imply a small fy value in equation (6.6). Low interactions also make it more 
likely that an early selection of solutions could be successfully combined. This 
latter fact leads to a small value in the factor in equation (6.6). 

To sum up, a “good” planning-domain decomposition should ensure that 
the number of operator-overlapping interactions and deleted-condition inter- 
actions between the decomposed domains are small. In this case a decom- 
position planner could still reduce the search time by decreasing both the 
branching factor of search and the depth of search. 

6.2.2 Backward-Chaining, Partial-Order Planners 

Korf ’s analysis did not take into account the possibility of a backward search. 
The analytical formula for a backward-chaining, partial-order planner is dif- 
ferent because, unlike forward-search planners, the branching factors are de- 
termined by the number of operators that can achieve a condition, rather than 
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the number of operators that are applicable to a state. The search depth is 
also different. The following analysis will be based on the analytical formula 
derived in Chapter 3. 

A. No Decomposition. We start by assuming that the goals are interacting. 

Suppose that for a goal pi, the branching factors are Bnew,i and Boid^ij rep- 
resenting the maximal number of new operators and existing plan steps that 
could achieve a precondition in a plan, respectively. Suppose also that to 
achieve a goal the optimal number of steps (discounting the start and 
final steps) is Ni. As in the forward-chaining case, we must account for the 
deleted-condition interactions and operator-overlapping interactions among 
the plans. The former adds a factor to the total solution length, while 
the latter subtracts a No factor. Let L be the sum of individual solution 
sizes: L = Ni. Recall that for backward-chaining, partial-order plan- 

ning, B^ = [{Bnew + Bold) * where t is the maximal number of threats 
in each plan, r is the number of ways to resolve each conflict, and P is the 
maximal number of preconditions for each plan step. Let B^ be the maximum 
such value among all goals. 

The time complexity can therefore be easily derived from equation (3.5) 
in Chapter 3: 

Time(No-Decomp) == O (6.7) 

B. Decomposition. With decomposition, partial-order planners follow exactly 
the same three stages as in the forward-chaining case. Suppose that, as before, 
each goal has k alternative solution plans. Let N be the maximal number of 
plan steps (except the start and the finish steps) in a solution plan for solving 
an individual subgoal pi. Then for backward- chaining, partial- order planners, 
using decomposition has a time complexity of 

Time(Use-Decomp) = 0(Time(GDECOMP) + 

[m * Bf^^ + 7 -(-^d+^o)] (6.8) 

C. Summary: Interacting Goals. When the goals are interacting, and when 
a backward-chaining, partial-order planner is used, we reach a similar, but 
not identical conclusion as in the total-order case. For decomposition to win, 
the number of operator-overlapping interactions and deleted-condition inter- 
actions must still be small. Low operator-overlapping interactions imply a 
small lo value in equation (6.8) and a small No in equation (6.7). Similarly, 
low deleted-condition interactions imply a small Id value in equation (6.8). 
Low deleted-condition interactions also increase the chance for an arbitrary 
selection of solutions to be successfully combined, thus reducing the kN factor 
in equation (6.8). 

However, we see that unlike the total-order planning case, planning both 
using and not using decomposition may have the same effective branching 
factors. This happens when the operator sets for the decomposed planning 
problems consist of the identical operators. However, a potential efficiency 
benefit for decomposition is likely derived from a reduction of the search depth. 
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6.2.3 Criteria for Good Decomposition 

A good decomposition should enable a planner to reduce the planning time. 
Based on the analysis for both total-order and partial-order planners, we see 
that when a “good” planning-domain decomposition is used, it is potentially 
possible for a planner to gain an exponential amount of efficiency by a re- 
duction of the search depth. For forward-chaining, total-order planners, an 
additional advantage is that the branching factor could be reduced. Prom 
both analyses, we see that a criterion for a good decomposition is that both 
operator-overlapping interactions and deleted-condition interactions be small. 

Note that for an arbitrarily chosen decomposition there is no guarantee 
that planning efficiency will be increased. Factors such as the number of 
operator-overlapping interactions and deleted-condition interactions play a 
central role in determining whether a decomposition is good. 



6.3 Goal-Directed Decomposition: 

The Gdecomp Algorithm 

Goal-directed decomposition refers to the task of decomposing a large set of 
goals into smaller subsets, such that each subset of goals is planned for to- 
gether, and that different subsets are planned for separately. We start design- 
ing the decomposition algorithm by re-visiting the intuition in the household 
domain. 

Suppose that the tasks for painting the house and furniture, and for doing 
grocery shopping, are represented as goals. One can easily imagine many such 
goals, the number of which can be quite large. Our intuition tells us that 
these goals could be roughly split into two groups, one for painting only and 
the other for shopping. The reason behind this grouping is that the actions 
relevant for the painting goals are highly related with each other, while they 
are less related to the actions relevant for shopping. Our conclusion here 
is that analysis of the potential interactions among the goals can provide 
sensible goal-directed decompositions. 

We try to capture this intuition in our decomposition algorithm Gdecomp 
(see Table 6.2). Below, we present this algorithm in a succession of steps. In 
the process, we assume that the definitions for all operators are given along 
with all constants in the domain. We assume that all operators are fully 
instantiated; the extension of the subsequent discussion to un-instantiated 
operators is straightforward. We also assume that the interactions among 
the goals are of a binary nature; only pairs of goals can interact. Finally, we 
assume that an initial state description and goals i = 1, 2 ... n are given. 

Step 1. Compute Relevant Operators. For a goal pi, only some operators are 
“relevant” to achieving it, either directly or by asserting the precondition of 
an operator which in turn is relevant to the goal. We capture this recursive 
definition for a relevant set of goal pi, as follows: 
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Table 6.2. The Gdecomp algorithm 



Input: A set of operators O, initial state init and goal g = ■ ,pn}- 

Output: A set of subsets, Qi, each consisting of the goals to be planned for together. 

Algorithm Gdecomp(0, init, g) 

1. For eaeh goal gi in goal set g, compute a set of relevant operators. 

2. for every pair of goals gi and gj, i ^ 

Estimate operator-overlapping interactions between gi and gj . 

3. for every pair of goals gi and gj^ i ^ j. 

Estimate deleted-condition interactions between gi and gj . 

4. Fix a problem-solving model (forward or backward), and construct 

a goal graph. 

5. Partition the goal graph, producing sets Qi,i = 1,2 ... ,m, 

6. return{{Qi,i = 1,2, . . .m}). 



Definition 6.3.1 (Relevant Operators) Let g be a goal and a £ O he an 
operator in a domain. The operator a is relevant to goal g, or a £ Rel{g), if 
and only if 

1. a directly achieves the goal: g £ Eff(a); or 

2. a achieves a precondition of some operator (3 £ Rel{g) . 

Steps 2 and 3. Estimate the Interactions. We consider each pair of goals 
gi and p 2 and estimate the number of operator-overlapping and deleted- 
condition interactions between them. One way to obtain such an estimate 
is using a library of past plans. By counting the average number of such 
interactions between previous solutions for gi and ^ 2 , a crude estimate could 
be obtained for Jo (pi, ^ 2 )- Ultimately a machine learning method should be 
used for computing these estimates. Based on previously generated plans for 
each goal, a prediction can be made for a future problem given that the 
planning-problem distribution stays the same. In subsequent discussion we 
assume that the estimates are known already. 

Step 4- Fixing a Planning Model: Forward or Backward. We must now de- 
cide on whether to use a forward-chaining or a backward-chaining method 
for solving the goals. A different method has a different computational com- 
plexity formula (See Chapter 3). For purposes of exposition, here we assume 
that a forward, total-order planner is used. 

We make the following assumptions: 

— let Bi be the branching factor of search for a subset of goals Gj; 

— let Di be the optimal solution length for 

— let B be the branching factor, and D the depth, of not using the decom- 
position in planning; 
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— let X{Gi,Gj) be the sum of all interactions between the members of the 
two goal sets; I{gi,gj) = Id(gi,gj) + Id(gi,gj) + Io{gi,gj); 

— let r be the maximal number of ways to resolve each conflict. 

We require that 

m m,m 

‘ +r^<B^, where I = Y. Gj ),i ^ j} (6.9) 

i=l = 

Under these conditions, it can be shown that a goal-directed decomposition 
planner has a lower worst-case complexity when (a) the first choice of plans, 
one for each goal set, can be consistently combined (A: = 1 in equation (6.8)) 
and (b) the computation for executing the G DECOMP algorithm is negligi- 
ble. Again, one should eventually obtain the above factors through machine 
learning. 

Step 4‘ Partition the Goal Graph. In this final step we use a greedy algorithm 
for performing the decomposition. This algorithm is based on a graph analysis 
using the estimated parameters. 

We first construct a graph of goals, where every node is a singleton set 
containing the goal. Between every two nodes is a weighted edge, where the 
weight represents the number of interactions between the pair; the weight 
between any two nodes gi and g 2 is I{{gi}, {52}) = Id{g\,g 2 ) + Id(g 2 ,gi) + 
Io{gi,g2)- 

If equation (6.9) is not yet satisfied, we wish to find a pair of nodes 
(ni,ri 2 ) such that if they are combined into a single node, the left hand 
side of equation (6.9) decreases. To decide which nodes to combine next, we 
assume that nodes ni and U 2 are chosen. Let the interaction measure between 
n\ and ri 2 be Z(ni,n 2 ), and the sum of the interaction measures among all 
the nodes be X. We would like to reduce the left hand side of equation (6.9) 
by a maximal number. Before the nodes n\ and ri 2 are combined, the size 
of the search tree expanded by a decomposition planner is YllLi 
After they are combined, the size becomes 

m 

-|_ ^Di+£) 2 ^(X-J(ni,n2)) 

i 12 

2=3 

where B \2 = {B\ 4- B 2 ) for forward, total-order planners. 

To select n\ and ri 2 , we require that the difference between the quantities 
is positive and maximal. This difference can be expressed by 

( 6 . 10 ) 

When all goals have the same branching factor and depth, this selection pro- 
cess is reduced to finding a pair of goals such that the number of interactions 
between them is maximal. 
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After Til and ri 2 are combined to form a larger node ni 2 = niUn 2 , the 
weights in the resulting graph should be recomputed to reflect the change. 
For all other nodes ns, the new weight between ni 2 and ns is the sum of the 
weights between n\ and ns, and TI 2 and ns- The process repeats until equa- 
tion (6.9) is satisfled. The resulting graph contains the flnal decomposition 
of the goals. 



6.4 Other Benefits of Decomposition 

We have argued about good decomposition from an efficiency point of view. 
In this section, we briefly mention some of the other benefits of goal decom- 
position. 

Concurrency in Plan Generation. By decomposing a problem domain into 
several parts, we could assign a planner to each part to generate a solution. 
The interactions among the parts could be represented as constraints among 
them, enabling more effective communications among planning activities [88]. 
Identifying Reusable Plan Parts. Suppose that an agent decides to reuse an 
existing plan. Suppose also that only a few goals in the current situation are 
different from the previous one. By having the domain decomposed we could 
quickly identify the parts of the plan affected by the change. In this case, 
only a limited amount of effort need be spent on finding sub-plans for the 
affected goals. 

Multi- agent Plan Execution. A solution plan to a decomposed planning prob- 
lem could be easily assigned to multiple agents to execute. The limited in- 
teraction among the agents ensures that there is a large degree of indepen- 
dence in each agent’s action. In addition, confronting the possible interactions 
among the agents also makes it possible to formulate communication channels 
between them. 

Balancing Achievable Goals. It is sometimes not economical or even possible 
to achieve all goals presented to a planner. In that case it is beneficial to 
identify a subset of goals that can be achieved. This decision can be made 
naturally when the goals are already decomposed into groups, so that between 
any two groups the interactions are limited. When many goals are to be 
achieved, the decision on which goals to keep and which to drop can be 
made more informative when the number of interactions among the goals is 
pre-estimated. 



6.5 Alternative Approaches to Decomposition Planning 

Divide- and-conquer has been a main theme of planning research. Although 
the main idea has always been to break apart a complex problem into simpler 
parts, the approaches vary drastically. Below, we consider some major thrusts 
in planning that takes decomposition into account. 
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Decomposition Along a Time Line 

The first work we consider is planning with abstraction. With this type of 
problem solving, a domain is broken down into several levels on a hierarchy. 
The most critical part of a planning problem is represented at the highest 
level, and a solution obtained at the lowest or concrete level is a solution to 
the original problem. 

Abstraction planning starts by finding a solution plan at the highest level 
and gradually refining the solution to each lower level. An early example of 
an abstract planner is Abstrips [113]. 

During planning, an abstract solution defines a collection of subproblem 
descriptions that are separated by time. The intuition is that most of these 
subproblems will not interact, allowing them to be solved more or less in- 
dependently. The abstraction paradigm was later extended in Knoblock’s 
Alpine [80], and Bacchus and Yang’s Hipoint [13], which will be discussed 
at greater length in Chapter 11. 

Despite the similarity in the general framework, major differences remain. 
The first difference is the manner in which a final solution plan is derived. 
In abstraction planning, a complete abstract plan is first found at the top- 
most level. This abstract plan ignores some less important preconditions and 
goals. Once found, it is taken down to a lower level where these other precon- 
ditions and goals are considered and achieved. This top-down process, called 
refinement, continues until a solution plan is found at the lowest level. 

During the refinement process, the abstract plan defines a decomposition 
of the original problem along a time line^ as shown in Figure 6.2. In this figure 
an abstract plan Av-^B^C is being refined at a lower level of abstraction. At 
the lower level, the abstract plan acts as a skeleton along the time line from 
the initial state to the goal state, such that between every pair of abstract 
plan steps is a new subproblem to be planned for. 

Decomposition planning, on the other hand, functions in quite a different 
manner. As shown in Figure 6.3 the planning process occurs concurrently^ 
instead of sequentially as dictated by an abstraction hierarchy. The solutions 
are formed in parallel, and later combined by interleaving operators and im- 
posing additional constraints. Therefore, when each sub-plan is formed, there 



Abstract Plan 



. ^-CSubprobleni' \ Subproblem p 



Fig. 6.2. Abstraction planning 
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Subproblem 1 



Subproblem 2 



Subproblem 3 






Fig. 6.3. Decomposition planning 



Combined final solution 

Ai ... C2 







is no elimination of preconditions as done in abstraction. The decomposition 
here is not along the time line, but instead, across the time line. 

A second difference between abstraction and problem-domain decomposi- 
tion, pointed out by Lansky[87], is that in obtaining the abstraction hierarchy 
used by Alpine and Hipoint, a requirement is enforced such that operators 
at a lower abstraction level cannot interfere with an abstract plan via interac- 
tions. This requirement is relaxed in decomposition planning, where we allow 
a non-empty set of interactions to occur between different groups of goals. 
These remaining interactions will be taken care of at a later stage, when the 
solution plans are combined. 



Decomposition in Strips Planning 

Strips is an early planning system designed to solve conjunctive goals [46]. 
Given a set of goals to achieve. Strips decomposes them into a sequence of 
subgoals. It then selects the first subgoal in the sequence, and finds an opera- 
tor to achieve it. The operator is likely to introduce new subgoals; those which 
are its preconditions. These subgoals are decomposed in the same manner. 
When a complete solution is found for a given subgoal. Strips executes that 
solution plan from the current initial state to obtain a new state in which the 
subgoal becomes achieved. 

Strips can be considered to implement a naive decomposition planning 
algorithm; its subgoals are solved one at a time and solutions to the subgoals 
appended in a linear sequence. However, when solving one subgoal. Strips 
does not completely separate it from the other subgoals. The effects of solving 
a subgoal is always felt by all the other subgoals. Nor does it decompose the 
initial state into subsets. The solutions to subgoals are only appended to each 
other. 

In contrast, the style of decomposition planning that we consider here 
separates the goals into subgoals as a result of analyzing the number of in- 
teractions among the goals. Thus, instead of always resulting in a decompo- 
sition where each subproblem contains only one conjunct, in decomposition 
planning each subproblem can be defined by multiplesubgoals that are more 
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closely related with each other than with the others. The conflict-resolution 
and solution-combination steps in decomposition planning are also more so- 
phisticated than a simple append operation. 

Decomposition in Partial- Order Planning 

Like Strips, most partial-order planning algorithms have taken a rather 
simplistic view of planning-problem decomposition, where a decomposition 
is based on individual conjuncts in the deflnition of goals and subgoals given 
in the input. Actions are added to a plan to achieve subgoals one at a time. 
When searching for a solution plan, the entire plan is still considered together 
for threat removal and step addition; the eflPect of achieving one goal has an 
immediate impact on the entire plan. 

A consequence of the partial-order planning approach is that no concur- 
rent problem-solving is done; the effect of any operator in a plan reaches 
across the entire plan. The problem-solving efficiency is gained only through 
the inherent partial-order and partial-binding representation of plans, but 
not from a reduction of the problem itself. 

A related effort in partial-order planning is the work on classifying 
problem-solvers by Barrett and Weld [17]. As a result of this analysis, partial- 
order planning is shown to be most effective in solving several types of inter- 
actions among goals and subgoals. An example of such situations is when the 
goals must be interleavingly achieved. However, the emphasis in this anal- 
ysis is not on decomposing a domain into concurrent parts, but rather on 
demonstrating that for a subclass of conjunctive goals partial-order planners 
are superior to some versions of backward-chaining, total-order planners. The 
work by Barrett and Weld can be seen as defining a set of detailed properties 
of subgoals as a function of planning algorithms. In contrast, our analysis pre- 
sented in this chapter can be seen as methods for automatically ascertaining 
some of these properties. 



Decomposition by Regions 

Closely related to our decomposition-planning algorithm are the Gemplan 
and Collage systems designed by Lansky et al. [88, 89]. This approach 
advocates an action-based planning framework by generating plans concur- 
rently based on regions of activity. The regions, which are either provided by 
the user or computed using a set of action constraints, function as a decom- 
position of the problem domain. The planning processes in different regions 
communicate with each other via user-defined constraints. Lansky refers to 
this style of concurrent planning by local regions as localization. 

Gemplan[88] and Collage[89] do not generate a decomposition of the 
problem domain into regions. Generation is handled by another system Log 
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[87], developed more recently. Loc takes as input a set of actions and con- 
straints in a planning domain and produces a hierarchical organization of re- 
gions. It accomplishes this task by invoking a procedure to determine whether 
two sets of actions are closely related; they are closely related when they are 
bound by the same constraint. For example, the actions paint -wall and 
repair-wall might be under the same constraint: before (repair-wall, 
paint-wall). In this case, the two actions belong to the same local region. 

In this manner, Loc determines a hierarchy of regions that forms a parent- 
child relationship. Planning can occur concurrently in different local regions, 
which are coordinated by inter-regional constraints. 

The Gdecomp algorithm is similar to Gemplan in spirit; they both 
implement planning as an instance of the divide-and-conquer methodology. 



Decomposition for Distributed Plan-Execution 

In distributed AI, a more recently developed group of planners are exclu- 
sively aimed at solving a complex problem by working on individual parts 
by multiple agents in a distributed way. Most of them, however, depend on 
the users to provide a decomposition before the algorithms can be used. One 
theme of work in distributed planning is to assign agents to tasks in a more or 
less optimal way. A characteristic example is the DMVT planner [41], which 
decomposes the agents’ environment by ranges of camera angles, assuming 
that corresponding to each sensor a dedicated agent exists. The Contract Net 
technique [125] decomposes a problem domain for a given set of agents, us- 
ing certain negotiation and bidding techniques. The aim in that work is to 
develop a common protocol to facilitate agent negotiations, rather than to 
decompose a domain using a centralized algorithm as we propose. 



6.6 Summary 

Divide-and-conquer is one of the problem-solving skills that an intelligent 
agent should possess. One instance of divide-and-conquer is to partition the 
goal set into a number of groups, such that the inter-group interactions are 
controlled. 

In this chapter we have analyzed the benefits of problem decomposition. 
This analysis confirmed our intuition that decomposition can lead to im- 
proved problem-solving performance. However, we have also shown how de- 
composition can hurt problem-solving if not used properly. It is the task of an 
intelligent decomposition method to make sure that the resultant decompo- 
sition is of good quality. We presented a criterion for judging good quality in 
terms of a complexity analysis. In addition, we discussed various approaches 
to decomposition and possible future extensions. 
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6.7 Background 



An early system that performed problem decomposition was Strips [46]. 
To solve a set of conjunctive goal literals, Strips takes every single goal 
literal and every subsequent precondition literal as a separate subgoal in the 
decomposed goal sets. The tradition was carried on to other types of planning 
method, including Waldinger’s planner [142], Nilsson’s DcOMP (see Chapter 8 
of [102]), and many partial-order planners (e.g., Tweak [28] and Snlp [17]). 
The Alpine system is described in [80]. 




7. Global Conflict Resolution 



In decomposition planning, a complex planning problem is split into several 
parts. Solutions to each part are generated and selected. When these disjoint 
solutions are combined to form a global solution to the original planning 
problem, conflicts between different sub-solutions will become explicit. These 
conflicts occur at this stage due to the assumptions made when solutions 
are derived for each subproblem. An example of such assumptions is one on 
resources; when generating solutions for a subproblem, one might assume that 
a certain resource is always available when in fact a solution for a separate 
subproblem also claims the resource. Because many conflicts may exist when 
the solutions are combined, it is necessary to design an intelligent conflict 
resolution method. This chapter presents a method for dealing with this 
problem. 



7.1 Global Conflict Resolution — An Overview 

Conflicts rarely exist alone. Therefore, conflict-resolution decisions should not 
be made in a local manner. To see this, consider a household example. 



An Example 

Suppose that one wants to paint a ceiling as well as a ladder. A proposed 
plan may consist of two parts, one for painting the ceiling and the other for 
painting the ladder, such that no ordering constraint is imposed between the 
two parts. If the robot hand can only hold one brush at a time, then a resource 
conflict occurs between the part of the plan that paints the ceiling and the 
part of the plan that paints the ladder, because of their competition for the 
robot hand. Similarly, if the wet paint from painting the ladder precludes one 
from climbing up, then another conflict occurs because performing the former 
negates a precondition of the latter, which requires that the ladder be dry. 

In the above example, a conflict is caused by a single plan step which can 
potentially delete a precondition of another step. The resource conflict can 
be resolved by painting the ceiling either before or after painting the ladder. 
And the wet-paint-on-ladder conflict is resolved by painting the ceiling first. 
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A successful resolution of both conflicts involves the recognition of several 
constraints on the order of plan steps and variable bindings. A consistent 
solution for resolving both conflicts is to paint the ceiling first. 

Overview of the Chapter 

In this chapter, we present a conflict resolution algorithm based on constraint 
satisfaction. This requires a formal representation of the individual conflicts 
and their resolution methods, an analysis of the relations among different 
conflicts, as well as the design of reasoning techniques that can facilitate 
global conflict resolution. We present a formalization of conflicts and their 
inter-relations, using an extended framework for solving constraint satisfac- 
tion problems, or CSPs (for an introduction to CSP, see Chapter 4). The 
formalization makes it possible to apply many existing techniques from the 
CSP area to aid efficient conflict resolution in planning. An additional feature 
of global conflict resolution is its extension to traditional methods for solving 
CSPs by employing a new constraint known as subsumption relations. These 
relations help remove a large portion of redundant constraints from the search 
space. Finally, we explore the utility of applying the CSP formalization to 
conflict resolution by pointing out where the proposed technique is expected 
to be most effective. At the end of the chapter we compare our method with 
other related work in conflict resolution. 



7.2 Conflict Resolution Constraints 

7.2.1 Conflicts and Conflict Resolution Methods 

In Chapter 2 we described two kinds of threats, positive and negative. We 
referred to the negative threats as “conflicts,” since they pose immediate dan- 
ger to the correctness of a plan. In this section we we develop a representation 
for conflict resolution. 

Let cl = {si ^ Sj) be a causal link in a plan, and t = {sk,e), where e 
is an effect of s/^, be a negative threat to cl (See Chapter 2 for a definition 
of negative threat). A conflict is defined as a pair conf (cl, t). We will refer 
to Si as a producer of p, Sj as the user of p, Sk as a clobberer and q as the 
clobbering effect. 

Suppose that the ordering relation in a plan iT is represented using a 
transitive closure; let TR{R) denote the transitive closure of a precedence 
relation R. Then for a given causal link cl in a plan, the set of all conflicts 
for cl can be found in time 0(n), where n is the total number of steps in the 
plan. 

For each conflict Conf = conf((s^ ^ Sj), (sfc,e)), let M(Conf) be the set 
of all resolution methods for resolving Conf. The set of alternative methods 
can be represented as a disjunctive set: 
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Table 7.1. Conflicts in the painting example. All conflicts have the same producer 
operator Si = Sinit 



Conflict 


Precondition 


User 


Clobberer 


Clobbering Effect 


Conf a 


Handempty 


get -brush (?c6) 


get-brush(?^6) 


-iHandempty 


Conf 6 


Handempty 


get-brush(?Z6) 


get-brush(?c6) 


-■Handempty 


Conf c 


Dry(?c6) 


get -brush (?c6) 


paint -ladder 


-Dry(?/6) 


Confd 


Dry(?/6) 


get-brush(?/6) 


paint-ceiling 


-<Dry(?c6) 


Conf e 


Dry(Ladder) 


paint - c e i 1 ing 


paint -ladder 


Dry(Ladder) 



M(Conf ) — {e ^ ~^p}} 

In actual implementation for generating the constraints, however, the to- 
tal number of conflict-resolution methods can be reduced by taking into ac- 
count the structure of the plan. For example, if the three operators Si,s^, 
and Sk are ordered in a linear sequence in a plan, such that the threat Sk is 
located necessarily between and , then only the separation methods are 
applicable for resolving the conflict without violating the existing ordering 
constraints. 

Consider the painting example introduced earlier. A plan for painting 
both the ceiling and the ladder consists of two unordered linear sequences of 
operators: 

“brush( ?c6) H^paint - c e il ing (Ceiling) 
»->return~brush(?c6)H-4s finish 

Sinit ^g 6 't“brush(?/ 5 )H-^paint-ladder (Ladder) 

^—>return-brush(?/6)^-^s finish 

In the above, Icb is a variable which stands for any ceiling brush, and ?lb 
is a variable that likewise refers to any ladder brush. The preconditions and 
effects of each operator in the plan are shown in Table 2.1, in Chapter 2. The 
conflicts in this plan are listed in Table 7 . 1 . 

The conflict resolution methods are: 

M(Conf a)= {{get-brush(?c6)H->get-brush(?^6)}, 

{return-brush(?/6)H-^get-brush(?c6) }} . 

M(Conf 5)= {{get-brush(?^fe)i-^get-brush(?c6)}, 

{return-brush(?c6)i-^get-brush(?/6)}}. 

M(Conf c)={{get~brush(?c6)t-^paint-ladder}, {?cb ^?lb}}. 

M(Conf d)={{get-brush(?/6)i-^paint-ceiling}, {?c6 ^?/6}}. 

M ( Conf e ) = { {paint -ceil ingi-^paint -ladder } } . 
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7.2.2 Relations Among Conflict Resolution Methods 

To find one or all of the consistent methods for resolving a set of conflicts 
in a plan iJ, one has to take into account the various kinds of relationships 
among different constraints. In this section, we define and analyze two types 
of relation, the inconsistency relation and the subsumption relation. 



Inconsistency Relation 

Resolving conflicts involves imposing constraints onto the structure of a plan. 
Some constraints cannot be imposed together, because they are inconsistent 
with each other. For example, imposing two ordering constraints a\^a 2 and 
a 2 ^a\ onto the same plan results in a cycle in step ordering, which is dis- 
allowed in a partial order. Likewise, constraints lx\ =7x2 and 7x2 ¥"7xs are 
inconsistent if the variables are already constrained in the plan such that 
7xi =7xs. 

With the transitive-closure representation TR for the step-ordering re- 
lation, inconsistency can be easily defined. Let Ri and R 2 be two sets of 
ordering constraints. Ri is inconsistent with R 2 in a plan U, or Iu{Ri, R 2 )i 
if and only if for some plan step s, 

(s, s) € Ti2(Order(77)URiUi^2)- 

That is, Ri and R 2 are inconsistent if and only if imposing both onto the. 
plan steps would result in a cycle. 

We can similarly define an inconsistency relation among two variable- 
binding constraints. Let Ri and R 2 be two sets of variable- binding con- 
straints. These constraints are either of the form 7x =?y, or of the form 
7x ^7z. The variable-binding constraints 7x =7y in both Ri and R 2 form an 
equivalence relation on the plan variables. An equivalence relation R on a, set 
S is defined as follows: 

— R is reflexive; that is, for every member a of i?, (a, a) € R. 

— R is symmetric; that is, for every pair of members a and b of R, if (a, b) £ R 
then (6, a) G R. 

— R is transitive; that is, for all members a, b and c in R, if (a, b) E R and 
(6, c) £ R then (a, c) G R. 

The first requirement in the above definition states that for every variable 
?x, 7x =7x always holds. The transitivity requirement states that for all plan 
variables ?xi, 7x2 and 7xs, if 7xi =7x2 and 7x2 =7x^, then 7x\ =7xs. Below, 
we consider the constraint = to form an equivalence relation. 
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Consider two variable-binding constraint sets R\ and i^ 2 - We say that, 
Ri is inconsistent with R 2 in a plan iJ, or In{Ri-,R 2 )i if and only if there 
exists a plan variable ?x, such that 

?x 7^?x G Binding(iI)Ui?iUi^ 2 - 

Two constraints, ordering or variable-binding, are consistent if they are 
not inconsistent with each other. 



Subsumption Relation 

Imposing one set of constraints may make another set redundant. For exam- 
ple, let Ri be {s 2 ^->S 3 }, and R 2 be {si^^ss}. If (sih->s 2 ) € Order (iJ), then 
imposing Ri would make it unnecessary to further impose R 2 - 

In general, Ri subsumes R 2 if imposing R\ will guarantee that R 2 is also 
imposed. Thus, R 2 is considered to be weaker than R±. More precisely, let 
Ri and R 2 be two sets of conjunctive ordering constraints. Ri subsumes R 2 
in plan 77, or Sn{Rh R 2 )i if and only if 

R 2 Q Ti^(i7iUOrder(7J)). 

Let Ri and R 2 be two sets of variable-binding constraints. R\ subsumes 
R 2 in plan 77, or Sn{Ri, R 2 )-, if and only if 

1. every variable-binding constraint ?x =?y of R 2 is a member of 77i, and 

2. every separation constraint lu ^?v of R 2 can be inferred from Ri and 
77. That is, for some variables ?x and ?y such that ?x =?u, and ?y =?v, 
we have 

(?x ^^y) € 7^iUBinding(77) 



As an example, let Ri = {(?x =?y)} and R 2 = {(?y ^?z)}. If (?x ^?z) G 
Binding(77) then Sn{RijR 2 ) holds. 

Given a plan 77, one can establish the subsumption and inconsistency rela- 
tions between any pair of constraints R\ and 7^2, by computing the transitive 
and equivalence closures of the ordering and variable-binding constraints, re- 
spectively. The former takes O(n^) time to compute for a plan with n steps, 
while the latter takes 0 {v^) time for a plan with v variables. 

For convenience, the subscript 77 used for both In and Sn relations is 
dropped in situations where it is clear about the plan under consideration. 

Because the subsumption relation S is defined via subset relations, it can 
be easily verified that S is transitive. That is. 

Lemma 7.2.1 If S{R\,R 2 ) and S{R 2 ,Rs), then S{Ri, R 3 ). 
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In addition, it is also easy to see that the following property holds: 

Lemma 7.2.2 If S{Ri, R 2 ), S{Rs,R 4 ) and I{R 2 , R 4 ), then I{Ri, Rs). 

This lemma states that if two constraints are inconsistent, then any stronger 
versions of the two constraints are also inconsistent. By letting Rs and R 4 
both be R', it holds as a corollary that if S{Ri,R 2 ), and I{R 2 ,R'), then 

Subsumption and inconsistency relations have so far been defined between 
a pair of constraints. Extensions to higher-order relations can be made in a 
similar manner. For example, two sets of precedence constraints R\ and R 2 
subsume Rs in a plan II if and only if Rs C Ti?(jRiUi? 2 UOrder(iI)). 

Minimal Solutions 

If C is the set of all conflicts in plan 77, and if a consistent set of constraints 
Sol resolves all conflicts in 71, then Sol is called a solution to C. 

It is possible to find a certain amount of redundancy in a solution. 
For example, suppose that in a plan 77 there is only one conflict, which 
could be resolved by either Sih->s 2 or lx ^ly. The stronger constraint 
Sol = {(sii-^S 2 ), {lx ^ly)} also resolves Conf . However, the latter is unnec- 
essarily strong, because either conjunct is able to resolve the conflict without 
the other. In this case, it is possible to reduce the solution Sol to a weaker one 
which is in some sense minimal A minimal solution Solmin for conflicts C is a 
solution for C such that no proper subset of Solmin resolves all conflicts in C. 

As the previous example illustrates, a solution may have several alterna- 
tive sets of minimal solutions; in the simple example they are soli = {S 11 - 4 S 2 } 
and S 0 I 2 = {lx ^ly}. If SoU is a minimal solution, and if SoV C Sol, then 
the constraints in the set difference {Sol — Sol') are considered redundant 
with respect to Sol'. As a consequence, if R\ and R 2 are two disjoint subsets 
of a solution Sol, and if Ri subsumes R 2 (i.e., S{Ri, R 2 )), then removing R 2 
from Sol leaves at least one minimal solution intact. This observation is the 
basis of a constraint-propagation rule presented in the next section. 



7.3 Conflict Resolution as Constraint Satisfaction 

In Chapter 4 we introduced constraint satisfaction as a mathematical and 
algorithmic tool for solving a class of combinatorial problems. In this section 
we will model conflict resolution as constraint satisfaction. 



7.3.1 Representation 

Conflict resolution in planning can be modeled by a CSP. For a given plan 77, 
a conflict Conf ^ in 77 corresponds to a variable. The domain of Conf ^ is the 
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set of alternative conflict-resolution methods that are capable of resolving the 
conflict. The constraints among the variables are defined via the consistency 
relations among different sets of conflict resolution methods. A solution to 
the CSP corresponds to a set of consistent resolution methods that resolves 
all conflicts in 77. 

An advantage of the mapping from conflict resolution problems to CSPs 
is that many existing strategies for solving general CSPs can be directly 
applied to facilitate a global analysis of conflicts. In addition, the existence 
of subsumption relations among the variables provides new opportunities for 
simplifying a CSP further than permitted by traditional CSP techniques. 

The methods for solving a CSP can be roughly divided into two categories: 
local constraint propagation and global, heuristically-guided backtracking al- 
gorithms. 

7.3.2 Propagating Constraints Among Conflicts 

When two or more variables are considered together, certain implicit con- 
straints among them can be inferred from the given ones. Consider, for exam- 
ple, a plan containing two conflicts, Conf^^ and Conf^, where the resolution 
methods have been found out to be 

M{Coh±a) = {{x = y, 6i-^c}, . . .}, and M(Conf b) = {{a: / y}, {c^b}}. 

From the inconsistency relation I{x = y^x ^ y) and 7(6t-^c, ci-^6), it is clear 
that the constraint set {x = y, b^c}, for Conf cannot be used as part of a 
solution for solving both conflicts. Therefore, it can be removed from the set 
of resolution methods for Conf a without affecting any solution. Furthermore, 
if it is also the only method for resolving Conf a, then the plan 77 corresponds 
to a dead end; it cannot be resolved by simply imposing constraints. 

The above example is an instance of a general procedure known as arc- 
consistency for solving a CSP; Chapter 4 gives a systematic review for these 
and other CSP techniques. 

7.3.3 Redundancy Removal via Subsumption Relations 

Recall that a conflict- resolution method Ri subsumes 7^2, if R\ is stronger 
than 772- The presence of this relationship makes certain constraints redun- 
dant. In this section, we consider two cases in which redundancy can be 
detected. 

Redundant Conflicts. Consider a plan 77 containing, among others, two 
conflicts, Confi and Conf 2 . Suppose that every set of constraints for Confi 
subsumes some constraint for Conf 2 - Because any solution for resolving all 
conflicts in the plan must resolve Conf i , every choice of a resolution method 
from M (Confi) must also resolve Conf 2 . This fact holds even when a con- 
straint chosen from M(Conf 2 ) is removed from Sol. Therefore, if the conflict 
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Conflict 1 Conflict 2 Redundant: remove from CSP! 






Fig. 7 . 1 . Conflict 2 can be removed from a CSP by Var-Subsumption Theorem 



Legend: 

Subsumption Relation: -n=- 

Inconsistency Relation: ► 



Q-- 

Q" 
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Conf 2 is removed from further consideration when solving the CSP, any min- 
imal solution to the modified CSP could still solve the original CSP. The 
argument is summarized in the theorem below. 

Theorem 7 . 3.1 (Var-Subsumption). Let II be a plan with a conflict set 
C. Let Conf 1 and Conf 2 be two conflicts in C. Let M(Conf 1) and M(Conf2) 
be conflict resolution constraints for Conf i and Conf 2, respectively. 

Suppose that 

Vi 7 i G M(Conf 1), 3 jR 2 ^ M(Conf2) such that 5 (Ri,i? 2 ) 

Then Conf 2 can be pruned from the CSP without affecting any minimal so- 
lution for C. 

In this case, we say that Conf 2 is redundant. The theorem is illustrated in 
Figure 7 . 1 . 

Pruning of redundant variables from a CSP reduces the size of the CSP 
and therefore can lead to improved efficiency in constraint reasoning. 

Redundant Constraints. Removal of redundancy in the above form only 
utilizes the subsumption information. When both inconsistency and sub- 
sumption relations are considered together, it is also possible to remove in- 
dividual redundant values from the domain of a CSP variable. 

Consider again a plan iJ containing two conflicts, Conf 1 and Conf 2. Sup- 
pose that there is some constraint set R2 in M(Conf2), such that for every 
method Ri in M(Conf 1), either 

1 , J(Ri, 7^2)7 i-^-5 ^1 is inconsistent with R2; or 

2 . 37^3 G Conf 2 where Rs ^ R21 such that S{Ri, Rs)- That is, R\ subsumes 
some other constraints in M (Conf 2). 

Given the above conditions, we could show that R2 is redundant. To see this, 
consider a solution for the CSP. In this solution there must be a constraint 
for solving Confi. Let this constraint be T^i. If Ri is inconsistent with 7^2 5 
then R2 cannot be part of the solution. If, on the other hand, one can find 
an 7^3 in M(Conf2) such that Ri subsumes 7^3, then R2 need not be part 
of M(Conf2) since R\ alone can resolve Conf 2. As a result, no matter what 
method is chosen for Confi, R2 will not be chosen for a minimal solution. 
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Conflict 1 Conflict 2 




Legend: 

Subsumption Relation: 

Inconsistency Relation: ► 



Redundant; remove from 
domain of Conflict 2 



Fig. 7.2. A conflict-resolution constraint of Conflict 2 can be removed from the 
domain of Conflict 2 by Value-Subsumption Theorem 



This means that R 2 can be removed from M {Coni 2 ) without affecting the 
set of minimal solutions for resolving all conflicts in 77. This conclusion is 
summarized in the following theorem. 

Theorem 7.3.2 (Value-Subsumption). Let II be a plan with a conflict set 
C. LetCon±i andCon ±2 he two conflicts inC, and let M(Conf 1 ) and M(Conf 2 ) 
be their corresponding sets of conflict-resolution constraints. Suppose that 
there exists some R 2 in M(Conf 2 ); such that for all R\ in M(Confi); either 

1. I{Ri,R 2 ), or 

2. 37^3 € M(Conf2). Rz ^ R 2 CL'nd S{R\,Rz). 

Then R 2 can be pruned from M(Conf 2 ) without affecting the set of minimal 
solutions to C. 

This theorem is shown pictorially in Figure 7.2. 

Augmenting Arc- Consistency. Removal of redundant variables or values 
in a CSP is called redundancy removal It can be used to augment a traditional 
arc-consistency algorithm, described in Chapter 4, in the following manner: 
every time a pair of variables (X, V) is examined in an arc-consistency algo- 
rithm, a check is made to verify whether the variable Y is redundant using the 
Var- Subsumption Theorem (Theorem 7.3.1). The variable can be removed if 
the test returns true. 

A second test, using the Value-Subsumption Theorem (Theorem 7.3.2), 
can be performed to check whether a value of Y is redundant. Every redun- 
dant value V can be removed from the domain of variable Y. 

For a given set of constraints, the computations of both inconsistency and 
subsumption relations have the same time complexity. These relations are 
computed only once when a CSP is first initialized. Furthermore, the aug- 
mented arc-consistency algorithm takes these two relations as inputs, and 
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considers pairs of conflicts for both of them. Therefore, the additional con- 
sideration of subsumption relations in the augmented algorithm increases the 
complexity of the original algorithm only by a constant factor. 

Redundancy removal can also be extended in a similar manner to augment 
path-consistency algorithms, which examine and infer inconsistencies within 
groups of three variables [91]. Such an extension is straightforward, and a 
detailed description can be found in [153]. 

Augmenting Forward- Checking. Forward-checking is a form of combin- 
ing partial arc-consistency and backtracking. A full description is given in 
Chapter 4. Here we discuss how to augment a key subroutine, FwdRevise, 
by taking into account the additional subsumption information. 

Recall that FwdRevise takes two arguments, a current variable assign- 
ment X = and a set representing the remaining CSP. FwdRevise per- 
forms constraint propagation through the un-assigned variables, by consider- 
ing pairs of the form (X = (T, {'yl, t?2, . . . vn}) ). In this pair F is a variable 

in CSP, and the set contains F’s domain values. FwdRevise checks each 
value of Y in turn: 

for each (F,M(F)) G CSP do 

if there is a value v G M{Y) such that (tx, v) is inconsistent, 
then delete v from M(Y); 

end if; 

if there is a value v G M(Y) such that u subsumes v, 
then delete Y from CSP; 

end if; 

end for. 



7.4 The Painting Example 

Consider again the painting problem described in Section 7.1. The conflict- 
resolution methods for resolving all conflicts in this problem have been for- 
mulated in Section 7.2. To facilitate understanding, we rename the conflict- 
resolution methods as follows: 

M(Confa)= {ai, a2}, M(Gonf6)= {6i, 62}, 

M(Confc)= {ci, C2}, M(Confd)= {di, ^2}, 

M(Confe)={ei}. 



In the above, ai corresponds to the first conflict resolution method for the 
conflict Confa, and the rest are likewise defined. These conflicts are converted 
to a constraint satisfaction problem, shown in Figure 7.3. 
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Fig. 7.3. A CSP for the conflict-resolution in the painting domain. The solid ar- 
rows correspond to inconsistency relations, and the dashed ones the subsumption 
relations 



The conflict resolution process is as the following: 

(1) We start with Conf g which has the smallest cardinality. The only choice for. 
Confe is inconsistent with the second constraint set, a2, for Confa. Therefore, 
the latter is removed from M (Confa) by arc-consistency. 

(2) Next, the only alternative left for M(Confa) is {ui}, which is inconsistent 
with the first choice for Conf^. Thus, due to arc-consistency M(Conf6) is 
reduced to {62}. 

( 3 ) The currently remaining constraint for Conf ^ subsumes some constraints 
for resolving Confa, Conf c, and Conf g- Thus, from the Var- Subsumption The- 
orem, all three conflicts become redundant and thus can be removed. 

( 4 ) The remaining value {62} for Conf 5 is inconsistent with the first constraint 
for Confer . Therefore the latter can be removed by arc-consistency. 

( 5 ) Finally, the CSP contains only two conflicts, Conf^ and Conf^, and the 
remaining constraints left in M{Confb) and M{Confd) are consistent with 
each other, and can be combined as a global solution to the CSP: 

return-brush(?c6)i-^get-brush(?^6), ?c6 

This solution is also a minimal solution. The resulting plan is formed by 
ordering all ceiling-painting operations to be before all ladder-painting oper- 
ations, and by making sure that the ceiling-painting brush is different from 
the ladder-painting brush. 
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7.5 When Is the Constraint-Based Method Useful? 

Basic planning methods introduced in Chapter 2 have dealt with the problem 
of conflict resolution in a local, incremental manner. In each planning iter- 
ation only a few conflicts are introduced. In these systems, constraint-based 
conflict resolution may not show its greatest potential. In contrast, we expect 
the global conflict-resolution technique to be most useful when a large number 
of conflicts exist, and when an arbitrary choice in conflict resolution methods 
is not likely to lead to a flnal solution. In a problem-decomposition framework, 
many conflicts are likely to occur at once when solution plans to different sub- 
problems are combined. Therefore, our constraint-based conflict-resolution 
technique is expected to be particularly useful for combining plans. Below, 
we consider a few other uses of this theory. 

Related to problem-decomposition systems, a second type of planner for 
which the CSP method may be useful is one that employs a task-network 
hierarchy introduced in Chapter 12. A task-network planning system starts 
with a set of subgoals, and reduces each one according to a library of pre- 
defined networks of sub-plans. Each sub-plan may also contain more detailed 
subgoals that can be further reduced. The system terminates when every 
remaining operator in the plan can be successfully executed. Examples of 
such systems are SiPE [147], Nonlin [132], and Deviser [140]. When these 
systems are applied to complex domains, each task reduction may introduce 
many new steps that interact, which may in turn cause a large number of 
conflicts to occur. 

In addition, the CSP method is expected to be useful in plan revision^ 
where the input is a used plan that is possibly incorrect, and the output is 
a modified version of the plan which fits a new situation. For example, in 
the PRIAR system of Kambhampati and Hendler [70], a previously gener- 
ated plan is first retrieved. The system then identifies those preconditions 
of the operators that are no longer established or conflicted in a new situa- 
tion, and proposes plan- modification operations for re- achieving them. The 
inserted new operators may render the plan possibly incorrect by creating 
new conflicts, and the number of such conflicts may increase with the num- 
ber of faults in the original plan. In this case, our CSP method for conflict 
resolution can fix the remaining conflicts and arrive quickly at a conclusion 
about the validity of the fix. 



7.6 The Combine Algorithm 

This section describes an implementation of the above conflict resolution 
techniques in the Combine algorithm. The input of Combine is assumed 
to be a possibly incorrect plan, that is, a partial-order plan for which some 
instance might be incorrect. The algorithm outputs a correct partial-order 
plan derived from the input plan, if a correct plan exists. 
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Table 7.2. The Combine algorithm 



Algorithm COMBlNE(i7) 

Input: A plan 77. 

External Functions: 

— FwdChecking(CSP) is the forward-checking algorithm, described in Chapter 4. 

— PartialArc(CSP) returns a CSP as a result of a partial arc-consistency oper- 
ation. 

— BuildLinks( 77) returns the plan with all causal links established. 

— DetectConfs(cZ, 77) returns the set of all conflicts with a causal link d in a plan 
77. 

— BuildCSP(cZ,77) constructs a CSP representation of conflict-resolution con- 
straints. 

— AppLYCoNSTRAlNTS(Solution,77) returns 77 upon which all constraints in 
Solution are imposed. 

Output: A correct version of 77, if one exists; Fail otherwise. 



1. if 77 does not already have causal links then 

2. 77 := BuildLinks(77); 

3. end if 

4. Conflicts := 0; 

5. for each causal link d in C-Links(77) do 

6. Conflicts := ConflictsUDETECTCONFS(c/, 77); 

7. end for 

8. CSP := BuiLDCSP(Conflicts, 77); 

9. CSP := PartialArc(CSP); 

10. if the domain of a variable in CSP becomes empty, then 

11. return(Fail) ; 

12. end if 

13. Solution := FwdChecking(CSP); 

14. if Solution ^ Fail then 

15. 77 := ApPLYCONSTRAlNTS(Solution, 77); 

16. return(77); 

17. else return(Fail); 

18. end if 



The Combine algorithm is shown in Table 7.2. The first task of Combine 
is to detect all conflicts with the causal links in the input plan. If the 
input plan does not come with causal links, then Combine will find the 
causal links in the plan by examining the producer-user relations (subroutine 
BuildLinks, Step 2). It does this by iterating through the preconditions in 
the plan, and for each precondition, it will look for a producer to establish 
it. If there are several alternative choices in producers, then one of them is 
chosen and the rest are stored as backtrack points. 
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After all conflicts are detected and recorded, Combine proposes a set of 
conflict-resolution methods as outlined in Section 2. The conflicts and the 
conflict-resolution methods form the basis for a CSP-representation. This is 
done in Step 8, in procedure BuildCSP. 

Once the CSP-representation is obtained, a partial arc-consistency algo- 
rithm is applied. This procedure performs two tasks: 

— using a partial arc-consistency algorithm to check for dead ends and to 
remove inconsistencies, and 

— using a redundancy-removal algorithm for eliminating redundant conflicts 
or constraints in the CSP. 

The partial arc-consistency algorithm checks, for every pair X and Y 
of variables in the CSP, whether a value of X is inconsistent with every 
value of Y. If so, then the value is removed from the domain of X. This 
is essentially the Revise algorithm of AC, which is described in Table 4.6. 
A difference from AC is that here we only perform Revise once through 
the constraint- network. Thus, it doesn’t re-check the consistency of X with 
other variables after an update in A’s domain. Although the partial arc- 
consistency algorithm does not ensure that the CSP network be completely 
arc-consistent, it does allow a significant number of inconsistencies to be 
removed. This implementation decision is based on the need to minimize the 
complexity of preprocessing algorithms. 

After partial arc-consistency is performed, a redundancy-removal proce- 
dure is followed. This procedure checks every pair of variables X and T, to see 
if every value of X subsumes some value of Y. By the Var-Subsumption Theo- 
rem, Y can be removed from the CSP if the condition is true. If so, then Y can 
be removed from the CSP. The Value-Subsumption Theorem is applied sim- 
ilarly for removal of redundant values. In this process, a count is maintained 
for the total number of times in which a value u of a variable X subsumes 
some value v of other variables in the CSP. The recorded measure s{u) will 
be used in the forward-checking as part of a variable-ordering heuristic. 

Finally, algorithm FwdChecking will be executed to find a set of consis- 
tent constraints for removing all conflicts. A subroutine of FwdChecking is 
VarOrdering(CSP), which sorts the variables of the CSP in the ascending 
order of their domain sizes. For variables of the same domain size, an option 
can be selected to order them in decreasing subsumption measure s(u), com- 
puted in the previous step. The purpose of this ordering process is for the 
backtracking algorithm to search a smaller search tree. During a backtracking 
process, an un- assigned variable can be removed from the remaining CSP if 
one of its values is subsumed by a previously assigned value. This enables the 
removal of redundant nodes as quickly as possible. 

When a consistent solution is found, it will be applied to U to generate 
a correct solution plan (Step 15). 

To evaluate COMBINE, we designed an artificially generated plan set with 
conflicts, on which we ran both Combine and a basic partial-order planner 
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Tweak (see Chapter 3) to compare their performance. This experiment is 
indicative of both systems when they are applied to the problem of plan- 
revision. The experimental results are discussed in detail in [154]. Here we 
summarize the main results: 

1. The Combine algorithm easily outperforms Tweak in time in this do- 
main. The performance difference increases as the number of conflicts in 
a plan grows. 

2. For plans containing conflicts that are not resolvable, Combine can de- 
termine that all resolution methods lead to failure more quickly than 
Tweak. On the other hand, Tweak has to go though an entire plan- 
ning iteration and could not utilize an intelligent order in which to resolve 
conflicts. 

3. The subsumption relation is more useful when conflicts are tightly cou- 
pled in a small portion of a plan, as indicated by the painting example. 



7.7 Related Work 

7.7.1 Overview of Previous Work 

Because of its importance in planning, methods for conflict resolution were ex- 
plored early on. Sussman’s system Hacker [130] recognized and fixed “bugs,” 
which were certain classes of conflicts. The bugs were fixed using a bag of 
hacks, which were different types of ordering constraints that could be im- 
posed upon the operators of a plan. Sacerdoti’s Noah [114] used a partial 
order to represent the structure of a plan, and implemented a more elaborate 
set of ordering constraints that could resolve different classes of conflicts. 
A problem with Noah is that given several alternative choices in the con- 
straints, it commits to one of them, and does not have the ability to backtrack 
should an inconsistent situation occur later. Tate’s Nonlin [132] fixed this 
problem and introduced a complete set of alternative ordering constraints 
that are capable of resolving conflicts in a completely instantiated plan. Rec- 
ognizing the need to represent resources using variables in a plan, Stefik’s 
Molgen [128] and Wilkins’ SiPE [147] both could further impose constraints 
on variable bindings to resolve conflicts. Chapman’s Tweak [28] introduced 
a white-knight constraint for resolving conflicts. Extending the operator lan- 
guage to more elaborate forms, Barrett and Weld [17] used confrontation to 
address actions with conditional effects. 

A major theme of the previous approaches to conflict reasoning is that 
conflicts are resolved incrementally and locally. In these systems, conflicts are 
reasoned about one at a time. In the presence of more than one conflict, no 
systematic theory exists to guide the conflict-resolution process. A drawback 
then is their loss of computational efficiency. A notable exception is the work 
by Joslin and Pollack [69], who take constraint-based conflict resolution as 
the basis of their algorithm. 
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7.7.2 Related Work by Smith and Peot 

A related work on conflict-resolution is that by Smith and Peot, who pro- 
posed a technique to postpone threat / conflict-resolution in a basic backtrack- 
chaining, partial-order planner. Their key observation is that certain threats, 
or groups of threats, in a plan do not have to be resolved as soon as they 
appear. The consideration of these threats can be delayed without sacriflcing 
the possibilities of flnding a solution. They argue that doing so has the ad- 
vantage of leaving a plan less constrained, enabling a planner to take more 
advantage of its partial-ordered nature. 

In this section, we show that the threat-analysis technique of Smith and 
Peot is in fact consistent with the formalization of conflict-resolution as con- 
straint satisfaction. Some of their threat-removal results are in fact derivable 
from arc-consistency properties, others are consistency properties relating N 
CSP-variables. To begin with, we first review their key results [124]. 

A fundamental concept in [124] is an operator graph. This is a directed 
graph representing the inter-relation between planning operators. In the 
graph there are two kinds of nodes, a precondition node for each precondition, 
and an operator node for every operator. Between a precondition node and 
an operator node there is a directed edge if the precondition is required by 
the operator. Similarly, if an operator has an eflFect which could unify with a 
precondition, then there is a directed edge from the former to the latter. In 
addition, each operator occurs only once in the graph. 

To illustrate. Figure 7.4 shows an operator graph for a simplifled painting 
domain. In an operator graph, an operator node a threatens a precondition 
node p if some eflFect of a unifies with the negation of p. By a convention used 
by Smith and Peot, a boldfaced arc denotes a threat (see Figure 7.4). 

Threats in an operator graph can be removed by imposing ordering con- 
straints between operators (for simplicity, in their exposition they temporarily 
ignored variable-binding constraints). In the example of Figure 7.5, adopted 
from [124], the threat between nodes 2 and B can be delayed because, no 
matter how the rest of the threats in a plan are resolved, there always re- 
mains a choice in which the operator node 2 can be ordered before operator 
node 1, which resolves the threat. 




Legend: 

Fig. 7.4. An operator graph for a simplified painting domain 
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Fig. 7.5. An operator graph showing how a threat, from node 2 to node B, can be 
delayed 



This observation is generalized by Smith and Peot as follows: Let T be 
the set of threats in an operator graph, and let P be a subset of those threats. 
The threats in P can be postponed if there is a set of ordering constraints 
that resolves the threats in P for every possible resolution of the remaining 
threats in T-P. 

This result is in fact a direct generalization of arc-consistency in a CSP. 
Consider the definition for an arc-consistency relation between two variables 
in a CSP. Recall that a pair of variables (X, Y) is arc-consistent if there is a 
consistent value in the domain of Y for every possible value of X. It is not hard 
to observe the parallel between the above theorem and the arc-consistency 
definition. In our formalization of confiict resolution as a CSP, every threat 
is a variable, and the threat-removal constraints are the variables’ values. If 
we use the structure of an operator-graph to define a consistency relation on 
operator ordering, such that no operator a is ordered both before and after 
another operator /3, then we have a CSP representation. A solution to the 
CSP is a set of consistent ordering constraints that could resolve all conflicts 
in the operator graph. From this view, we could obtain the same conclusion 
as that by Smith and Peot. 

In Constraint Satisfaction area, Freuder [53] has shown that when (a) the 
constraint relation in a CSP is binary, (b) the constraint network is a tree, and 
(c) the variable pairs are arc-consistent, the CSP can be solved in a backtrack- 
free order (see Theorem 4.5.1 in Chapter 4). This backtrack-free order is one 
in which the variables are instantiated, such that X is instantiated before Y 
only if (X,F) is arc-consistent. 

Appl}dng this result to the threat-removal problem studied by Smith and 
Peot, we observe the following. Let (T-P) represent an aggregate variable in 
a CSP, and P represent another. An aggregate variable is formed by repre- 
senting all threats in the set (T-P) as a single variable, and the Cartesian 
product of all constraints, as long as they are consistent together, as the 
values of the variable. Then Freuder’s theorem can be directly applied: The 
CSP is obviously binary since there are only two variables. For the same 
reason, it is a tree network of variables. Finally, we assume that this CSP 
is arc-consistent; every value of (T-P) leaves a consistent value for P. Then 
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by Freuder’s backtrack- free order, (T-P) should be solved before P — this is 
essentially the result of Smith and Peot as we described above. 

From the above discussion, we see that the CSP formalization of conflict- 
resolution problems not only ojQFers computational advantages, but also serves 
as a unifying framework in which some related work in this area, such as the 
one by Smith and Peot, could be understood from a new angle. 

7.7.3 Related Work by Etzioni 

Etzioni studied conflicts in the context of a total-order planner PRODIGY [25]. 
In his Static system, the operator deflnitions are compiled into an AND/OR 
tree. Based on this AND/OR tree Static extracts useful information about 
necessary subgoal clobberings and compiles this information into search- 
control rules. The rules are then used during the operation of a total-order 
planner to sequence the goals in order to avoid certain predictable subgoal in- 
teractions. It was shown that these rules are as eflFective as those learned dur- 
ing planning by explanation-based learning (EBL) systems such as Prodigy. 

A diflPerence between Static and our framework here is that Static com- 
putes all necessary interactions between pairs of subgoals. These are the in- 
teractions that are guaranteed to occur as long as the two subgoals appear 
simultaneously in a plan. In our case, however, we compute all possible con- 
flicts among subgoals, anticipating what might occur during planning. These 
conflicts are used globally to obtain a conflict resolution strategy. 



7.8 Open Problems 

Temporal Operator Language 

One advantage of our theory is its extensibility; with a more elaborate plan- 
ning language, the underlying theory for global conflict resolution need not 
change. For example, one can extend the partial-order planning language to 
include the time-point algebra of Vilain and Kautz [141], by associating the 
occurrence of each action with a time point. One can also extend the partial- 
order planning language to include Allen’s interval representation of actions. 
With Vilain and Kautz’s time-point logic, the relationships between two time 
points include “precedes,” “follows,” “same,” and “not-same.” With the new 
language, one can also augment the set of conflict resolution methods by 
providing an additional set of constraints. For example, suppose that when- 
ever two operators occur simultaneously, one of their combined effects will 
clobber an establishment relation. Then one way to resolve the conflict is to 
impose a “not-same” constraint onto the tim^ points of the two operators. 
This augmentation only enlarges the domain of individual variables that rep- 
resent conflicts in a CSP, and thus the same computational framework can 
be directly applied to resolve conflicts in the extended language. 
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Domain-Dependent Conflict Resolution 

So far we have considered conflict resolution only in the context of generat- 
ing a correct partial-order plan. Conflict resolution, however, has much wider 
application ranges. For example, in electronics design, one has to place a 
set of electronic devices onto a small board area. Some of the devices might 
have adverse properties such as generating heat and magnetic-electric inter- 
ferences. Each of these undesirable features, which can be formalized as con- 
flicts, could be removed in a variety of ways, ranging from adding a physical 
shield to moving the interacting devices spatially apart. To find a consistent 
combination of ways for removing all conflicts requires a similar technique to 
what we described here. 

Another example where the problem of conflict resolution often occurs 
is in social-political situations involving multiple social groups. This is es- 
sentially a multi-agent situation. Each group might represent a variety of 
objectives, many of which are often in conflict. To resolve these conflicts, 
one could employ a number of well-known methods, such as commissioning 
a go-between or offering some kind of concession. 

In both these areas the conflict-resolution constraints are dramatically dif- 
ferent from those we consider in this chapter. They are all domain-dependent 
in nature. However, we stress here that despite the differences in the nature 
of conflict-resolution constraints, at a sufficiently high level, many conflict- 
resolution problems could still be formalized and solved using the constraint- 
based technique that we have proposed in this chapter. 



7.9 Summary 

In this chapter, we have described a theory of conflicts and conflict- resolution 
methods in planning. In this theory, a conflict is modeled as a variable in 
a CSP the set of conflict-resolution methods is modeled as the domain of 
a variable. Two types of relations are described. The inconsistency relation 
corresponds directly to its counterpart in CSPs, and the subsumption relation 
provides new insights into the removal of redundancy values and variables. 
The formalization supports a number of efficient reasoning tasks, including 
arc-consistency enforcement, redundancy-removal, dead-end detection, and 
the ordering of conflicts in which to conduct their resolution. 

In contrast to the incremental conflict-resolution methods used by a basic 
planning algorithm, a global conflict-resolution algorithm offers the following 
advantages. 

Recognition of an Intelligent Ordering of Conflicts. In the presence of a large 
number of conflicts, the order in which the conflicts are resolved may have 
dramatic effects on efficiency. The global constraint-based method presented 
above allows the recognition of these orderings to be made. A conflict ordering 
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heuristic can be further strengthened by recognizing “redundant” constraints, 
which exist because some conflict resolution constraints may be stronger than 
others. For example, for a set of conflicts C, there may exist a subset «S of C so 
that once conflicts in S are resolved, all other conflicts in C are also resolved. 

Early Dead-end Detection. A plan in which conflicts cannot be resolved to- 
gether corresponds to a dead end in a planner’s search space. The ability to 
detect such dead ends early is vital to a planner’s efliciency. In many situa- 
tions, unresolvable conflicts can be detected with relatively low costs when 
considered together, but they may not be obvious when only a single conflict 
is considered at a time. Thus, a planning system based on the incremental 
method for conflict resolution may incur expensive computation due to the 
expansion of plans that eventually leads to dead ends. 

Preprocessing of Constraint Network. Considering conflicts in a global man- 
ner permits the application of arc-consistency algorithm to the conflict set, 
which may lead to the discovery that some constraints can never participate 
in any final solutions. 



7.10 Background 

Conflict resolution has been a central problem in planning. Its development 
therefore followed a steady pattern of evolution; the reader could trace much 
of the historical background in many foundational planning papers. These 
include [142], [130], [128], [132], [146], [28], [94]. 

The material described in this chapter was adapted from an Artificial In- 
telligence journal article by Yang [154], which in turn evolved from an AAAI- 
90 paper that proposed to use an algebra of conflicts for planning [151]. The 
implementations of the COMBINE algorithm applied a method by Hertzberg 
and Horz for classif 3 dng conflicts [65] . Ephrati and Rosenschein considered a 
similar conflict-resolution problem in the context of distributed, multi-agent 
planning [42]. The Static system is described in [44], Prodigy in [25], and 
Smith- Peot method in [124]. 




8. Plan Merging 



Problem decomposition can improve planning efficiency by partitioning a 
complex problem into simpler parts. Since the sub-problems are solved sep- 
arately, this planning strategy can also incur redundancy. To remove the 
redundant parts of a plan, some plan-steps can be merged^ producing a plan 
that is less costly to execute. 

This chapter provides a precise definition for plan merging and presents 
optimal and approximation algorithms for finding minimal-cost merged plans. 
The optimal plan-merging algorithm applies dynamic programming to find 
a minimal-cost merged plan from a partial-order plan. To cope with plans 
with large sizes, we also present an approximation algorithm along with an 
analysis of the quality of its output, in worst and average cases. 



8.1 The Value of Plan Merging 

The value of utilizing operator-overlapping interactions, or helpful interac- 
tions, was recognized early in AI planning research [114, 132, 147]. A helpful 
goal interaction occurs in a plan, or among several plans, if it enables a reduc- 
tion in plan costs. An important type of helpful goal interaction occurs when 
certain operators in a plan can be grouped, or merged^ together in such a way 
as to make the resulting plan more efficient to execute. This happens often 
in domains where redundant setup and restore operations can be eliminated 
in the execution of consecutive tasks and where redundant journeys can be 
eliminated by fetching multiple objects at once. In the related work section, 
we review another use of plan merging, for speeding up the reasoning process 
for planning, through a notion known as overloading [108]. 

In Chapter 6 we have shown a plan merging problem in the painting 
domain. Here we consider a story given by Wilensky [145]. 

John was planning to go camping for a week. He went to the super- 
market to buy a week’s worth of groceries. 

The main character in this example, John, had a set of subgoals to achieve, 
each subgoal being to buy food for a meal during the camping week. How- 
ever, instead of making several shopping trips separately for each individual 
meal, John was able to merge the plans for the subgoals, and achieve them 
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Fig. 8.1. A variable-sized blocks world 



simultaneously with a single trip to the market. The resultant merged plan 
is more efficient to execute than the separate ones. 

Merging actions for reducing plan costs, as demonstrated above , is typ- 
ical of the kind of optimization task people do for general transportation 
problems. For example, suppose that cargo items A and B both need be de- 
livered from location L\ to location L 2 by planes. If the two delivery goals 
are planned separately, then two airplanes would be required for both A 
and B. For each item, separate loading and unloading operations are needed. 
However, if A and B fit into one plane, then combining the two sub-plans 
can produce a more efficient overall plan for both goals. This merged plan 
requires one combined loading and unloading operation for A and B, as well 
as a single flight operation. The result is a plan that is considerably cheaper 
than the original ones. 

Plan merging is equally important for minimizing the costs of robot task 
plans. As an example, consider a blocks world problem with blocks of different 
sizes. To pick up one block of a certain size, the robot arm has to mount a 
gripper of an appropriate size. Suppose that only one robot arm exists, and 
in order to grab a block of a different size, the robot has to unmount the 
current gripper and mount the gripper with the new size. In this case, it is 
more efficient to group block-stacking operations that use the same type of 
grippers. An example is shown in Figure 8.1. 

The blocks world problem is illustrative of the role of operator merging in 
robotic-assembly domains. Identical plan-merging issues also arise in the do- 
main of automated manufacturing where process plans for metal-cutting [76] , 
set-up operations [63] and tool-approach directions [93] need to be optimized. 
Similarly, in the area of query optimization in database systems [117], as well 
as domains having multiple agents [41, 111], plan merging in multiple plans 
seems inevitable. 

In developing a computational theory on plan merging, we address the 
following questions: 

1. What are the conditions under which plan steps in a plan can be merged? 

2. What is the computational complexity of optimal plan merging? 

3. What are the optimal algorithms for plan merging? When are these al- 
gorithms feasible in practice? 
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4. Where the optimal algorithms are infeasible to apply, what are the ap- 
proximation algorithms that can perform efficiently? What is the quality 
of the plans produced by these algorithms in worst and average-cases? 

Our approach to addressing the above questions is to combine formal 
methods developed in Artificial Intelligence and computational techniques 
from Operations Research (OR). In particular, we first propose a formaliza- 
tion of plan merging using the STRIPS operator definitions. Based on the 
formalization, we present a dynamic programming algorithm for determining 
the optimal solution. While the dynamic programming algorithm has tra- 
ditionally been formulated for inputs corresponding to matrices of symbols, 
AI planning has been concerned about plans represented as partially-ordered 
operator sets. To bridge the two different fields, we also extend the dynamic 
programming method to handling partially-ordered plans in a novel way. 

One drawback of the dynamic programming method is that it becomes 
computationally infeasible for problems of larger sizes. While we are able to 
phrase an optimal algorithm for general purpose, domain-independent plan 
merging, its run-time requirements may be prohibitive for inputs of large 
sizes. To make the planning problem more tractable, most existing systems 
that consider helpful interactions employ certain kinds of greedy algorithms 
for plan merging [63, 114, 132, 147]. Thus, we also describe a specification of 
an approximation algorithm for merging plans, and analyze the quality of its 
outputs in worst and average cases. The approximation algorithm has linear 
time-complexity in the number of plan steps of the input plan. 



8.2 Formal Description 

8.2.1 A Formal Definition for Plan Merging 

Recall that a plan II consists of a set of steps related to each other via a set 
of constraints. These steps are instantiations of a set of operator schemas. 
For simplicity, we first assume that all operator instantiations and steps are 
fully instantiated; they don’t contain variable parameters. 

For a given plan iJ, there may be some steps in the plan that can be 
grouped together and replaced by a less costly step that achieves all the 
useful effects of the grouped steps. In such a case, we say that the steps are 
mergeahle. 

The notion of merging plan steps can be illustrated graphically in Fig- 
ure 8.2. In this figure, there are two plans to be combined, Planl and Plan2. 
The steps S2 and T2 can be merged together if there exists an operator M 
which can replace the collective functions of these two steps, where M is less 
costly than the sum of 52 and T2. M provides the functions of steps 52 and 
T2 if it can replace the two steps in their respective causal links. 
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mergeable steps 




Fig. 8.2. Merging plan steps 



We now make the concept of plan merging more precisely. We say that a 
set of steps U in a. plan II can be grouped together if no other step outside of 
E is ordered between any pair of steps in E. More precisely, 

Definition 8.2.1 (Step Groups) 

A set of steps E in a plan II can he grouped together, if 

Vsi,Sj € E,~3sk G {Steps{TI) — E) such that Si^-^Sk^Sj. 

As an example, the set of steps {52, T2} in Figure 8.2 can be grouped to- 
gether, whereas {51,T2,53} cannot. 

A set of steps that can be grouped together can behave like a single step. 
Let E he a set of steps that can be grouped together. Some effects of the 
steps in E are useful in 77, because they establish the preconditions of some 
other steps that are outside of E. Other effects are simply side-effects of the 
steps in E and do not serve any purpose as far as the correctness of 77 is 
concerned. We use Useful-Effects (Z*, 77) to denote the set of all useful effects 
of the steps in E. Likewise, we use Net-Preconds(Z, 77) to denote the set of 
all preconditions of the steps in E not already achieved by any steps in E. 
More formally, 

Useful-Effects (Z, 77) = 

Userje G Eff(s) I 3s j € (Steps(77) — Z). {s Sj) e C-Links(77)}, 
Net-Preconds(Z, 77) = 

^seuiP ^ Pre(s) | 3si G (Steps(77) - Z). (s^ ^ s) G C-Links(77)}. 

As an example, let Z be {52, T2}, and our plan be the one shown in Fig- 
ure 8.2. Then Useful-Effects(Z, 77) = and Net-Preconds(Z, 77) = 

{p,u}. 

Given a set Z of steps, suppose that they can be grouped together. Z is 
mergeable if Z as a whole can be replaced by a cheaper step without affecting 
the correctness of the plan. More precisely, Z is mergeable in 77 if there exists 
an operator /i such that 




8.2 Formal Description 



125 




Fig. 8.3. A blocks world plan 

1. E can be grouped together in il, 

2. Pre(/i) C Net-Preconds( iJ), and Useful-Effects ( 17, il) C 

That is, the step can be used to achieve all the useful effects of the 
steps in E while requiring only a subset of their preconditions. 

3. Cost(/x) < Cost(i7). 

The step fi is called a merged step of E in the plan U. Because a set of steps 
can potentially be merged and replaced by several alternative operators, each 
with its own cost and precondition and effect sets, we denote the set of all 
merged operators by Merge jj- (17) (or simply Merge(17) if it is clear about 
the plan U). 

8.2.2 An Example 

The definition for step merging covers the examples given in the previous 
section. Below, we illustrate the definition through the blocks world example. 

Consider again the multi-gripper blocks-world problem with blocks of 
different sizes (see Figure 8.1). A plan for solving the goals is shown in Fig- 
ure 8.3. Suppose that can-grab-A is a precondition of the step moveA, and 
can-grab-B is a precondition of the step moveB. Then the following causal 
links establish precondition-dependencies between the steps in the plan: 

(moimt -gripper- A ^ moveA ( Al, Ta6/e)), where p\ = can-grab-A 
(mount -gripper- A ^ moveA(A2, Ta6/e)), 

(mount -gripper-B ^ moveB ( 1, Al)), where p 2 = can-grab-B, and 
(mount -gripper-B ^ moveB (jB2, A2)), 
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Fig. 8.4. Merged blocks world plan 



The mount -gripper- A step in this example involves removing any previously 
mounted gripper and mounting a gripper of a new size. In the plan, the two 
moimt-gripper-A steps can be merged into a single instance of the step. So 
can the moimt-gripper-B steps. This can be verified by the following facts: 

1. The two mount -gripper- A steps can be grouped together in the plan, 
since no other steps are necessarily between them. 

2. The set Useful-Effects ({mount -gripper-A, mount -gripper-A}) is exactly 
{can-grab-A}, which is identical to the effect of a single mount -gripper-A 
step. Similarly, the subset condition for net-preconditions is also satisfied. 

3. Cost(moimt-gripper-A) < 2 * Cost (moimt -gripper-A). 

After a similar merging of the two mount-gripper-B steps, the plan after 
merging is shown in Figure 8.4. 

There are several ways to extend the definition for plan merging. For 
example, we could allow the merged operator to assert only a subset of 
the useful effects of the steps in as long as the rest of the merged plan 
could restore the missing effects. We do not delve into the detailed case study 
of such a definition, but instead allow the readers to modify the definition 
depending on their particular problem domain. Our algorithms below could 
be easily adapted to these new definitions. 

8.2.3 Impact on Correctness 

Suppose that a set E of steps are merged into an operator p,. In addition to 
the useful effects of /i, there may also be other side effects that p achieves. 
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Fig. 8.5. An example where merging helps conflict resolution 



Some of these side effects might be harmless, others might work against the 
correctness of the resulting plan by threatening an existing causal link in 
the plan. Thus, in deciding whether to merge a set of steps into a particular 
operator, cost is not the only issue; plan correctness is a factor that must also 
be taken into account. 

To preserve plan correctness during merging, we could establish certain 
conditions for the merged operator to satisfy. We require that no side effects 
of /i negate the causal links of the resulting plan. This condition is stated 
formally as follows: 

Condition [Correctness Preserving]. An operator fj, = Merge jj{E) sat- 
isfies the correctness-preserving condition if 

1. For every causal link (s^ Sj) G C-Links(i7), if can possibly be be- 
tween Si and Sj, then for every effect e of that is not a member of 
Useful-Effects( i7, il), e does not unify with ->p. 

2. The preconditions of ^ is a subset of Net-Preconds(i7, 77). 

When this condition holds and when the set E is replaced by fi in the plan, 
all preconditions of /i are already satisfied and all causal links established by 
members of E are preserved. In addition, no side effects of p pose threats 
on an existing causal link. The resulting plan is therefore still correct. It can 
further be shown by induction that the correctness of a plan is preserved 
by merging steps in a plan any number of times, as long as the correctness- 
preserving condition is satisfied. 

Plan merging can not only reduce plan costs, but also be used to resolve 
conflicts in a plan. As an example, consider a plan for painting both a ceil- 
ing and a door (shown in Figure 8.5). The literal p in the figure states the 
condition that the ladder is available. On the left-hand side of the figure, 
we see two plans, each with a separate get-ladder step conflicting with a 
get -ladder step in the other plan. The conflict arises because after getting 
the ladder for executing one painting plan, the agent did not specify releas- 
ing the ladder in order for the other plan to be executed. After merging the 
two actions, we obtain a plan on the right-hand side of the figure, where the 
conflict disappears. 

This example shows that plan merging can help improve the correctness of 
a plan by reducing the number of conflicts. This additional method of conflict 
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resolution by plan merging is important especially when plans are developed 
separately. It is not hard to see how we can incorporate it as an additional 
conflict-removal method in the constraint-based technique we discussed in 
Chapter 7. 



8.3 Complexity of Plan Merging 

Two important issues arise in plan merging. The first is deciding what steps 
to merge in a partial-order plan. The second is how to merge them. Below, 
we analyze these two problems separately. 

8.3.1 Deciding What to Merge 

Deciding what to merge may be computationally difficult if no additional 
domain knowledge is given. This is because for a given plan, it might be 
necessary to examine the useful effects and net preconditions of every subset 
of steps, and the number of such subsets can be large. To make the pro- 
cess more efficient, various kinds of domain knowledge can be employed. For 
example, one way for the steps in E to be merged is when they contain var- 
ious sub-steps which cancel each other out, in which case the merged step /i 
would correspond to the set of steps in E with these sub-steps removed. In 
manufacturing domains where it is desirable to minimize set-up costs, this 
situation occurs often and can be profitably employed [100]. This case also 
corresponds to what Wilensky calls “partial-plan merging” [145]. Another 
case is when all the steps in E share a common schema, and in this case, the 
goals of these steps can be achieved by executing the schema only once. This 
case corresponds to what Wilensky calls “common-schema merging.” 

To make the problem sufficiently concise, we make a number of assump- 
tions. 

Operator-type Assumption. We assume that sufficient domain knowledge is 
available about what steps can be merged, and for each set of these steps, 
what the merged steps are. In particular, we assume that each operator 
schema in a domain description is associated with an operator type. For exam- 
ple, in the blocks world domain, a gripper-changing operation for switching 
to a particular gripper has a type which corresponds to the size of the grip- 
per. As another example, in the painting domain all get -brush operators 
might be associated with the same type. Given this typing knowledge, we 
concentrate on the second issue, that of finding and analyzing methods for 
computing an optimal plan. 

Unique Merged- operator Assumption. Given that the mergeable steps are 
known in a plan, one more complication still exists. For a given set E of steps 
to be merged, there may be several alternative merged steps, {//i, . . . , /x^}, 
to choose from, each with a different set of preconditions, eflPects and cost 
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value. To further remove this complication, we assume that for each set E of 
mergeable steps, there is a unique merged operator /i. This restriction can be 
easily relaxed by slightly augmenting the dynamic programming algorithm 
introduced in the next section. 

Mergeability Assumption. We assume that two or more plan steps can be 
merged only if they are unordered with each other. If they are ordered so that 
one is before another, then in fact the subsequent algorithms only require 
minimal change. However, to keep our model clear and concise, we carry 
through our presentation with this assumption. 



8.3.2 Deciding How to Merge 

Not all mergeable steps can be simultaneously merged. The partial order 
defined on II steps may render some pairs of mergings inconsistent; merging 
one set of steps may make merging others impossible. For example, consider 
the two plans 

and A2^B2^C2, 

where the plan steps can have three types. A, B and C. Thus, after step 
merging, one merged plan is 

Merge({^1, A2})i-»52i-^Merge({C'1, C2])^Bl, 

while another merged plan is 

Merge({^1, A2})h->C'1h->Merge({B1, B2))^C2. 

However, it is impossible to merge both pairs B1,B2 and d,C2 without 
violating the partial order (see Figure 8.6). 

We now consider the computational complexity as a result of the above 
complication. The problem is to decide which set of mergeable steps to merge, 
when a partial order prevents all of them from being merged simultaneously 
We show that the problem of finding an optimally merged plan is NP-hard. 



Merge({A1 ,A2}) Merge({C1 ,C2}) Merge({B1 ,B2}) 




Fig. 8.6. Computational complexity of plan merging 
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We first introduce the Shortest Common Supers equence (SCS) problem: 
given m sequences of characters , 52, . . . , 5m, what is the shortest sequence 
5 such that each Si is a subsequence of 5? This problem has been shown to 
be NP-hard [55]. 

We give a transformation of a SCS problem to a plan-merging problem. 
In a SCS problem, for each sequence Si = (c^i, Ci 2 , • • • , we define a 
plan Pi = (flii, • • • , where all steps have the same costs. We also 
define a goal Gi achieved by Pi. The only kind of interaction that occurs 
is an operator-overlapping interaction; we define two steps Oij and aki to 
be mergeable if and only if Cij = Cki. It follows that the plans Pi,..., Pm 
can be merged into a plan P = (ai, U 2 , . . . , a^) if and only if the sequence 
5 = (ci, C 2 , . • • , Cfc) is a supersequence for 5i , . . . , 5m- 

One way to get around the NP-hardness of a computational problem is 
to set an input parameter constant. In many plan-merging problems this 
strategy could be applied by setting the total number of goals to be achieved 
constant. Thus, for these naturally occurring cases the number of input plans 
is also constant. In such cases, the optimal solution to the SCS problem can 
still be found in polynomial time using a dynamic programming algorithm. 



8.4 Optimal Plan Merging 

We now describe how to find an optimally merged plan using dynamic pro- 
gramming. A set of partially ordered plans can be combined into a single plan 
il, the steps of which correspond to a network of nodes, where each node is 
a plan step, and an edge between two nodes is a precedence relation between 
the corresponding steps. Figure 8.7 shows an example of viewing a three-plan 
merging problem as a network problem. 

The intuition behind dynamic programming is inductive computation. 
One starts from a small subset of the plan steps and computes an optimal 
merged plan for the subset. Then one can extend the subset by including 
more steps, and compute the cost of the merged plan for the enlarged subset, 
based on both the cost of each smaller subset, and the costs of the new plan 
steps. For an introduction to dynamic programming, see Chapter 4. 

We consider the process of computation as one of a left-to-right push of 
a frontier which divides the plan into two parts. The part on the left denotes 
a subset of the plan steps for which an optimal merged plan has already 
been found, and the one to the right those plan steps that are yet to be 
included. The frontier includes all nodes (or plan steps) that are unordered 
with respect to each other in the plan, and that are being considered for 
merging (see Figure 8.7). Let F be the set of nodes on the frontier. Let Si 
be the set of all nodes on the frontier F and whose corresponding plan steps 
have the same type, Type(i),^ = l,2,...,m. These are nodes that can be 
merged together on F. Finally, let di be a subset of Si,i = 1,2, .. .m. Then 
if Optimal(F) denotes the cost of the optimal merged plan for the subset of 
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F 






Fig. 8.7. Intuition of dynamic programming 



plan steps upto and including the frontier F, the cost can be formulated as 
follows: 

{ Optimal(F — (Ji) + Cost(^i), Vcri C 
Optimal(F - + Cost(M 2 ), C 

Optimal(F - am) + Cost(/i^), C 

In the above formula, fii is a merged operator for set cr^, i = 1,2, ...,m 
respectively. 

Having designed a method for computing the cost of the optimal merged 
plan, we can then generalize the computation so that along each frontier, the 
optimal choice in step-merging, a, is remembered. The sequence of merged 
steps, as we push the frontier F from left to right, will give rise to a set of 
plan steps that should be merged to result in an optimal merged plan. 

More specifically, for each frontier F, let cr be a set of merged steps on 
F which gives rise to a minimal value as computed by equation (8.1). Let 
Merged-Steps(F) be the remembered subsets of plan steps up to and includ- 
ing the frontier F that should be merged. We have 



Merged-Steps(F) = Merged- Steps (F — cj)U{cr} 
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To apply the dynamic programming algorithm, Dynamic, of Chapter 4, 
we must specify a parent-child relationship between states. In our formula- 
tion, each frontier F is a state. A parent of F is a frontier E such that the 
“difference” between E and F is a set of plan steps a to be merged. Let a be 
a subset of F. Let /? be a set of plan steps such that 

-- every member of (3 is immediately ordered before a member of cr. A plan 
step Si is immediately before Sj if in the plan there are no other steps that 
are ordered between and Sj . And 

- every member of /? is unordered with every other member of (3 and every 
member of F — cr. 

Consider the largest such f3, and call it Patch(cr). Patch(cr) can be considered 
as a patch for the “hole” created by cutting a set a from the frontier F. Given 
the set Patch((j), a parent state of F is 

P = {F — cj)U Patch(cr) 

The set of all parents of F, Parents (F), is the set 

Parents(F) = {(F — (j)uPatch((j) | Vcr € i = 1, 2, . . . , m} (8.2) 

This set represents various ways of arriving at the current frontier from all 
previous ones. 

Finally, let d(F, F) be a decision in the dynamic programming algorithm 
(see Chapter 4, Table 4.4), for going from a parent state F to a child state F, 
via merging of a set a of plan steps. This is denoted as Merge(ct). Having 
these functions in place, a dynamic programming implementation of optimal 
plan merging is simply a function call Dynamic{{s finish}) ^ We will refer to 
this instantiation of the dynamic programming algorithm for optimal plan 
merging as OPTiMALMERGE(iJ). 

The dynamic programming algorithm is listed in Table 8.1. 

Now we consider the complexity of the algorithm. Let k be the maximum 
size of a frontier; k determines how “fat” the partial-order plan is. If the plans 
are all linear sequences of steps, then k is the number of goals to be achieved, 
which we represented by m in Chapter 6. 

To carry out the computation for equation (8.1), it is necessary to enu- 
merate all subsets of the mergeable steps on a given frontier. This takes time 
0(2^). Then the OptimalMerge algorithm iterates through all frontiers in 
the plan, the total number of which is where I is the maximal length of 17. 
Here the length of a partially ordered plan is defined as the maximum number 
of steps that form a linear chain from start of the plan to finish. Thus, the 
total amount of time taken by the algorithm is 

Time(OPTlMALMERGE) = 0{2^l^) 

This formula shows that when the width of a plan 17 is fixed at a constant, the 
time complexity of the OptimalMerge algorithm increases polynomially 
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Table 8.1. The OptimalMerge algorithm 



Algorithm OPTlMALMERGE(iJ) 
Input: A plan U. 

Output: An optimal merged plan. 



1. call UPTOFRONTIER({s/inis^}); 

2. for each set a in Opt-Decisiom{{s finish }) do 

3. merge a in plan iJ; 

4. end for 

5. return(il); 



Subroutine UptoFrontier(F) 

Input: A search frontier F. 

Output: The cost of an optimal merged plan up to the frontier. A global variable 
Opt-Decisions(F) records all steps that should be merged. 



1. if F is the start step Sinit then 

2. return(O); 

3. else 

4. Parent-List := Parents(F), calculated using Equation 8.2; 

5. let 5 be a parent of F in Parent-List such that 

Cost := (Cost(MERGE(cr)) -f UptoFrontier(S)) is minimal; 

6. Opt-Decisions(F) := Opt-Decisions(5)u{(j}; 

7. end if; 

8. return(Cost); 



with the length of plans. This indicates that the algorithm is most useful 
when the plan is a small collection of almost totally ordered plans. In fact, 
this dynamic programming algorithm is the basis for merging two sequences 
of text strings in UNIX systems using “diff.” However, when k becomes large, 
the computation can become very expensive. 



8.5 Approximate Plan Merging 

For problems of large sizes, the complexity of the dynamic programming 
methods may be too high to be practical. An alternative choice is to use ap- 
proximation algorithms that operate in low-order polynomial time but output 
a subopt imal merged plan. 

In the past, such planning systems as Noah [114], SiPE [147], and Ma- 
chinist [63] have resorted to the application of greedy algorithms for plan 
merging. However, the approximation algorithms used by the systems were 
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never clearly specified, nor was it known how these algorithms perform in the 
worst and average cases. In this section, we consider a greedy algorithm that 
is intuitively appealing, and present complexity analysis on its performance. 

8.5.1 Algorithm ApproxMerge 

Recall that the dynamic programming algorithm Optimal Merge compares 
all possible merges on all frontiers in the plan. This is in fact one source of in- 
efficiency. In the approximation algorithm, we simplify the OptimalMerge 
algorithm by 

— considering only a limited number of frontiers; and 

— considering only a limited number of sets of steps for merging, along a 
frontier. 

As with OptimalMerge we assume that our plans are arranged in a left 
to right order, and we push a frontier. Start (77), in that direction. Once a set 
of steps is decided to be merged on a frontier, we take that merging operation 
as finals and take the steps off the plan. This forms a new frontier. In this 
way, every step is considered by the approximation algorithm only once, and 
thus the time complexity of the algorithm is linear in the total number of 
steps in 77. 

To make the above picture more precise, we formally define a frontier of 
a plan 77 as Start (77). Given a plan 77, one can find a subset of steps with 
no other preceding steps. Formally, 

Start (77) = {s G Steps(77) | -•3s^ G Steps(77). s^h^s} 

In addition, let 17 be a set of plan steps in 77. The function Remove(I7, 77) 
returns the plan 77 with all steps in E removed. 

The approximation algorithm is shown in Table 8.2. It classifies all steps 
on the current frontier as denoted by Start (77) by operator types. It then 
merges all steps in a subclass Ei with maximal cardinality. 



8.5.2 Example 

We trace the ApproxMerge algorithm on the plan in Figure 8.6. For sim- 
plicity, we assume that the cost of a set of steps is the cardinality of that 
set. Recall that in this example, any steps with the same alphabet can be 
merged. 

Initially, Start (77) = {sinit}- After the start step is removed from the 
plan, the current frontier contains Ea = {A\, A 2 }. These two steps can be 
merged, giving rise to a current record of mergeable steps Merged-Steps = 
{{^1. ^2}}. 

After Ea is removed from the plan (Step 5 of the algorithm), the current 
frontier consists of {C'i,jB 2 }. An arbitrary choice selects Ec — {C\}. After 
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Table 8.2. The ApproxMerge algorithm 



Algorithm ApproxMerge(77) 

Input: A plan 77. 

External Function: ApplyMerge(5, IJ) returns IJ with the steps in S merged. 
Output: A merged plan. 



1. Merged-Steps := 0; IJ' := iJ; 

while 77 0 do 

2. partition Start(77) into m subsets XJi by type; 

3. let Ej be a set such that 

(Cost(I7j) — Cost (M ERG e(I7j))) is maximal; 

4. Merged-Steps := Merged-StepsUlITj}; 

5. TJ := Remove(Z'j, 77); 

6. end while 

7. return(ApPLYMERGE(Merged-Steps, 77’); 



Ec is removed from the plan, Start(77) = {Bi, B 2 ], which can be merged 
into a B step. After the last step C 2 is removed from the plan, only the finish 
step remains, and the algorithm terminates. The merged plan is 

MERGE{{Ai, A2})^Ci^MERGE{{Bi, B2})^C2 

8.5.3 Complexity 

Since Algorithm ApproxMerge considers each step only once, its time com- 
plexity is linear in the number of steps. We are hence more interested in the 
quality of the plan it produces in worst and average cases. 

For ease of analysis of the worst case complexity in plan quality, we assume 
that the plan 77 consists of a set of totally ordered plans. Also for simplicity 
we assume that the cost of a plan is the number of steps (excluding the start 
and finish steps) in the plan. 

Theorem 8.5.1 (Worst Case). Given a set ofn totally ordered plans, each 
of length n, the worst-case cost 0/ ApproxMerge is 0{nlogn). 

Proof. We first give a set of n plans each containing n steps of type in 
{a,/3}, from which algorithm ApproxMerge will produce a superplan of 
size i?(n log n). For operator of u > 2 types, the proof is similar. We conve- 
niently arrange the plans into an n by n matrix M such that each row of M 
corresponds to a plan, one step per entry. We recursively construct M: Its 
first Ln/2J -1- 1 rows are all a. The first column of the rest of the rows con- 
tains /3. Therefore, according to its description, algorithm ApproxMerge 
always chooses the first column of the first [n/2j +1 rows to merge at the 
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first n steps. At step n + 1, ApproxMerge chooses the first column of 0 
from the rest of rows. After the execution of the first n -f 1 steps, we are left 
with a matrix of roughly [n/2j — 1 by n — 1. If we recursively apply above 
construction to the remaining matrix, it is apparent that the merge process 
wiU go on for i?(n log n) steps. 

We now show that ApproxMerge always produces a super-plan no more 
than size 0(n log n) on plans drawn from a binary type for plan steps (for 
fixed number of operator types u > 2 the proof is similar). The minimum 
number of steps possibly merged by ApproxMerge in successive steps is 
a monotonically decreasing function f{k) of step number k. The worst case 
behavior of ApproxMerge maximizes the step number at which f{k) van- 
ishes. (We approximate f{k) by its continuous analog f{x).) All realizations of 
f{x) have f{x)dx = n^, that is, ApproxMerge merges all steps into 
a super-plan. As well, f { x ) > (2n)~^ f{y)dy^ for ApproxMerge is able 
to merge at least half of the number of plans not yet exhausted by step fc. The 
worst case behavior of ApproxMerge is obtained by setting f { x ) to its min- 
imum value for all x. In this case, /(O) = n/2 and f { x ) = (2n)“^ f{y)dy^ 
with solution f{x) = (n/2)e“^/^’^. For large n, this function first vanishes 
when X = cn log n for some c > 0. □ 

We now examine the average case cost of ApproxMerge. We assume 
that the average case is one in which every operator is equally likely to ap- 
pear in a plan. Under this uniform-distribution assumption, we can employ 
a probabilistic analysis of the algorithm. The proof, which we omit here, is 
a bit involved. Interested readers could consult [49]. We provide the final 
conclusion in the theorem below. 

Theorem 8.5.2 (Average Case). Given n randomly generated, totally or- 
dered plans, each containing at most n steps taken from an operator set 
{ai,a 2 , . . - , otu}} CL'^d for any small positive constant e, the average case cost 
o/ A pproxMerge is no greater than n{u -f l)/2 -1- logn), where u 

is the total number of mergeahle operator types. 

The proof of this theorem assumes that the plan sizes n are asymptoti- 
cally large. This theorem established that the linear-time algorithm Approx- 
Merge is in fact expected to perform well when mergeable step types are 
uniformly distributed over a plan and when the plan sizes are large. 



8.6 Related Work 

8.6.1 Critics in Noah, Sipe, and Nonlin 

Sacerdoti’s Noah[114] system is one of the first planners to explicitly seek 
opportunities for plan merging. It relies on its set of critics to handle possi- 
ble interactions among the different parts of a plan. Three critics are intro- 
duced for the purpose of improving the quality of plans, including “eliminate 
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redundant preconditions,” “use existing objects” and “optimize disjuncts.” 
The “'eliminate redundant preconditions” critic can be considered as merging 
two or more steps that are used to achieve the same preconditions, as long as 
no precedence relation in the plan is violated. Wilkins’ SiPE [147] and Tate’s 
Nonlin [132] are other planning systems that have relatively more advanced 
capabilities in step merging. For example, they are able to recognize that a 
goal is achievable by an existing step in the current plan. In such situations, 
they will impose constraints on orderings and variable bindings so that the 
step is used to achieve it. This process, known as phantomization of a goal, is 
also used in Kambhampati and Hendler’s plan-reuse framework for reducing 
plan cost [73]. At a meta-planning level, Wilensky [145] considers different 
types of positive goal relationships^ in the context of cognitive modeling of 
human problem solving. 

Despite early recognition of its importance, none of the previous work 
has actually considered a formal, precise characterization of plan merging. 
Nor are algorithms for optimal and approximate plan-merging presented or 
analyzed. Our formalization and computational framework for plan merging 
could be considered as a formal basis for these systems. 

8.6.2 Machinist 

In contrast to the above domain-independent approaches to planning, Hayes 
[63] proposed a domain- dependent method for plan merging in machining 
domains. Through careful cognitive studies in the manufacturing domain, 
Hayes observed that many expert machinists often look for opportunities for 
machining-operator overlap during a plan-generation process. In this domain, 
each machining operator is described by an orientation of the part, fixtures 
for the part (such as clamping), and tool types (drilling, milling, etc.). Each 
machining feature, a hole or a pocket, could be considered as a goal, and a 
sequence of machining operations a plan. Because different features on the 
same part might involve the same fixture and tools, and because some of 
these identical fixture and tool operations could considerably shorten a plan, 
an expert machinist will not only be on the lookout for possible operator- 
overlaps, but also plan for these overlaps to happen. 

Her observation is implemented in a rule-based system called Machinist. 
In the system an approximation algorithm is implemented for performing the 
merging operation. The results are compared with that by human machinists 
in terms of the quality of the plans produced, and the amount of time it 
takes to develop a plan. It was shown that Machinist can often out-perform 
an experienced human process-planner. For example, the quahty of a plan 
produced by Machinist was shown to be much better than one by an expert 
machinist with three years of experence, and only slightly worse than one by 
a machinist with five years of experience. 
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8.6.3 Operator Overloading 

Many of the formalization and computational considerations in this chapter 
are centered around a single notion: to make a plan less costly to execute. 
Plan merging, in fact, has been seriously considered by Pollack [108] and 
others as a machanism for speeding up the plan-reasoning process, through 
a notion known as overloading. 

In [108], Pollack presents an enlightening example to illustrate the over- 
loading concept: consider a plan for getting flour for baking a cake. Suppose 
that, at the moment, the agent has a number of active plans, already de- 
veloped for some other goals such as to go to a bank. By recognizing the 
opportunities for merging the plans for flour-shopping and visiting the bank, 
the agent could choose to overload the banking plan to accomplish both goals. 
An end efiFect is reduced planning time for solving all goals. 

As Pollack pointed out, the concept of overloading is supported by some 
strong cognitive evidence [64]. To implement overloading requires that we 
modify our problem-decomposition framework. Instead of generating alter- 
native plans for each goal before merging them, one could interleave plan 
generation and plan merging, in order to take immediate advantage of over- 
loading a plan. In this way, the plan found for all goals may not be the 
absolutely optimal one, but for a rational agent, as Simon argues in [122], 
the satisficing approach might very well be the most effective way for dealing 
with limited computational resources. 



8.7 Open Problems 

8.7.1 Order-Dependent Cost Functions 

One observation by Hayes in the manufacturing domain is that the cost of a 
plan is often dependent on the order in which the steps in a partial-order plan 
are sequenced. This presents a new problem in plan merging, as in the entire 
chapter we have assumed that the cost of a plan is the sum of all step-costs, 
independent of the order. When knowledge is available on how the cost of a 
plan changes with step-ordering, both the dynamic programming method for 
optimal plan merging and the approximate plan merging method could be 
adapted for merging plans. 



8.7.2 Hierarchical Plan Merging 

Real-world problems are inherently hierarchical. Often one could separate the 
most important aspects of a complex problem from the rest. One criterion 
for such separation is cost. In a machining domain, Britanik and Marefat 
(see [22]) noticed that operations on a machined part could sometimes be 
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partitioned into classes, one for fixturing, another for tooling. The fixture- 
setup operations usually take up so much time that these operations could 
be placed at a higher level of abstraction. With this hierarchical knowledge, 
they proposed an algorithm for merging plans in a hierarchical manner; plan 
merging for the fixture-setup operations is done first, that for tooling next, 
and so on. It is shown in [22] that this approach could dramatically reduce the 
time needed for performing plan-merging, while not sacrificing much of the 
plan quality. It would be interesting to see how this approach of hierarchical 
plan merging could fit under a similar formal framework for plan merging, 
and to study under the framework the benefits of the approach. 

8.7.3 Enhancing Conflict Resolution 

In this chapter we have also shown how plan-merging could be used as 
a method for conflict resolution. Although we have exposed the intuition 
through an example, we have not developed a formal model to incorporate 
this technique. We have instead taken the conceptually clearer approach of 
separately discussing plan merging and conflict resolution. It would be worth- 
while to see how plan-merging and conflict resolution could be fruitfully com- 
bined in a seamless manner. 



8.8 Summary 

Opportunities for plan merging are particularly abundant when each plan 
is generated separately from the rest. In this chapter we have presented a 
formal model for plan merging, showing the precise meaning of merging steps 
to reduce plan costs, and demonstrating when such operations will not affect 
the correctness of a plan. We have also provided an optimal algorithm and 
an approximation algorithm for computing a merged plan. 



8.9 Background 

Most of the background material has been covered in Section 8.6. The work 
presented in this chapter is adapted from three sources: Yang’s PhD disser- 
tation [150], an Artificial Intelligence journal article by Foulser, Li, and Yang 
[49], and part of a Computational Intelligence journal article by Yang, Nau, 
and Handler [158]. The dynamic programming algorithm is made consider- 
ably more concise than the one presented in [49]. 
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In this chapter we deal with the problem of selecting plans from a set of 
pre-existing alternatives. We consider two closely related versions. The first 
problem occurs when plans are selected based on a set of consistency criteria. 
The second problem arises when plans for different goals can be merged to 
improve efficiency. 



9.1 Consistency-Based Plan Selection 

So far we have discussed how to resolve conflicts among separately generated 
plans using constraint satisfaction. In many real-world situations, planning 
problems can be decomposed in such a way that for each sub-problem, there 
is more than one plan. Thus, in addition to the problem of combining the 
solutions, there is also a problem in selecting one plan per sub-problem, in 
order for them to be combined. In this section, we describe another constraint- 
based method for multiple-goal plan selection. 

9.1.1 The Multiple-Goal Plan- Select ion Problem 

The multiple-goal plan- selection problem occurs in several practical planning 
domains. One example is database query-plan generation. Typically in the 
database domain, a query could be answered by alternative query plans, each 
being a sequence of database access operations. To answer multiple queries, 
an intelligent database-management system selects one query plan from each 
set and combines them together. 

We demonstrate the multiple-goal plan-selection problem via a version of 
the painting domain. Suppose our tasks are to paint a ceiling, a ladder, and 
a door. Suppose also that for each goal, we have multiple alternative plans 
to choose from. In particular, we have 

goali = Painted(Ceiling): 

Till = machine - s and ( Ceiling) i-^spray”paint( Ceiling), 

IIi2 = hand- s and ( Ceiling) H^spray“paint( Ceiling), 
iJi3 = whit e-paint (Ceiling)H-^hand-paint (Ceiling). 
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goal 2 = Painted(Ladder): 

11 21 = machine-'Sand(Ladder)H-^spray-paint (Ladder), 

11 22 = liand-*sand(Ladder)H-^spray-paint (Ladder), 

11 23 = whit e-paint (Ladder) t->hand-paint (Ladder). 
goals = Painted (Door): 

IIsi = machine-sand(Door)H-^spray-paint(Door), 

IIs 2 = hand-sand(Door)H-> spray-paint (Door), 

LTss = whit e-paint (Door) t-^hand-paint (Door). 

Suppose that not all alternative plans can be consistently combined; some 
might be incompatible. For example, it might be the case that iJn and II 2 s 
cannot be used in the same global plan due to a certain side effect of spray 
painting. In such cases, choosing which plan to use for each subgoal is a 
non-trivial task. 

9.1.2 A CSP Representation 

We now formalize the problem. Suppose that we are given a set of decomposed 
goals gi,i = 1, 2, . . . , n, and a set of alternative plans {IIij,j = 1,2,... m}, 
one set for each goal Our problem is to find one plan for each goal such 
that all selected plans can be combined into a correct global plan. 

We model this problem as a constraint satisfaction problem. In particular, 
let every goal set gi be represented as a variable, every alternative plan II ij 
as a value in the domain of A consistency relation Consistently is defined 
between two values A == II ij and B — Ilik such that 

Consistent jj( A, R) = TRUE iff Combine( { il/A:}) / Fail 

where Combine() is a procedure (Table 7.2) of resolving conflicts in a plan. 
See Chapter 7 for more descriptions. 

In general, the relations between the variables in this CSP are not just 
binary; three or more values can be related to each other via a consistency 
relation. This feature calls for a general backtrack-based method where in 
every step of search, all sub-plans accumulated so far are checked against 
a global consistency relation. This would ensure that the final global plan 
is conflict-free. Nevertheless, using the binary consistency relations defined 
above we could prune away many incompatible values before the backtrack- 
based method is used. 

9.1.3 A Constraint-Based Solution 

As with many other constraint-satisfaction problems, our solution to the 
multiple-plan selection problem can be formulated in two phases, local- 
consistency methods and intelligent backtrack-based methods. Our adapta- 
tion of these methods is implemented in a planning system known as Wat- 
PLAN. 
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Watplan first applies local-consistency methods in an attempt to prop- 
agate consistency constraints among subsets of plans. Arc-consistency is ap- 
plied to pairs of goals. When a plan II ij is found to be inconsistent with 
all plans for another goal, Ilij can be pruned away. Similarly, we could de- 
fine a subsumption relation between plans, such that a plan Ilij subsumes 
TIuv if the former could solve the goal , of the latter. With subsumption 
relations so defined, we could then apply the Var-Subsumption and Value- 
Subsumption Theorems defined in Chapter 7 to prune away redundant goals 
and plans. 

Freuder’s theorem (Theorem 4.5.1) also opens up an interesting opportu- 
nity for efficiently solving the multi-goal plan-selection problem. For problem- 
decomposition, one might discover that the interaction between the goals 
defines a tree-shaped network. If one can further ensure arc-consistency by 
studying pairs of frequently used plans for each goal-set then by Theo- 
rem 4.5.1, we know that a backtrack- free order of goals can be followed to 
form a global, consistent plan for all goals. 

In general, however, we may not be so lucky as to have a tree-shaped 
decomposition of the domain. In such cases, once arc-consistency is processed, 
we can then use a backtrack-based method, such as forward-checking, to 
compose a global solution. The resulting algorithm Watplan is shown in 
Table 9.1. 

The core of WATPLAN is a forward-checking algorithm, with a slight mod- 
ification. As described in Chapter 4, forward- checking performs a backtrack- 



Table 9.1. The Watplan algorithm 



Algorithm Watplan(5, V) 

Input: A set of decomposed goals Q = {gi^i = 1,2,...}, each goal is associated 
with a set of alternative plans V = {Pi}i where each Pi = {Ilij}. 

External Functions: 

— FwdChecking(CSP) is the forward- checking algorithm, described in Chapter 4. 

— PartialArc(CSP) returns a CSP as a result of a partial arc-consistency pro- 
cessing. 

Output: A global, consistent plan Solution for solving all the goals, if one exists. 
Otherwise Fail. 

1. represent the problem {Q, V) as a CSP; 

2. let the constraint satisfaction problem be CSP; 

3. CSP := Partial Arc(CSP); 

4. if a variable in CSP becomes empty, 

5. then return(Fail); 

6. end if 

7. Solution := FwdChecking(CSP); 

8. return(Solution); 
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ing search with partial look-ahead. During this process, a binary consistency 
relation between any pair of plans could be verified between the current vari- 
able assignment and future ones. Two plans ili and II 2 are consistent if 
CoMBiNE({iIi, 772}) does not fail, where the Combine algorithm, defined in 
Section 7.6, performs conflict resolution for the two plans. Once a complete 
assignment is accumulated, with one value for every variable, Watplan then 
checks whether all plans can be consistently combined, using the Combine 
algorithm. If not, backtrack occurs. 

9.1.4 Evaluation — a Blocks- World Example 

As predicted before, a global analysis of conflicts is expected to be particu- 
larly useful for problem solvers that are based oii a problem-decomposition 
strategy. A situation in which problem-decomposition can be profitably ap- 
plied is where there is enough domain-dependent knowledge for generating 
solutions to each individual sub-problem, but the conflicts among the sub- 
solutions need be resolved when a global solution is formed. Watplan is 
extended to interface with a set of specialists to facilitate this way of prob- 
lem solving. In particular, it is assumed that an ordered set of alternative 
solutions has been generated by each specialist within the problem domain. 
Watplan then conducts a systematic selection of the sub-solutions, and ap- 
plies its conflict-detection, preprocessing, variable-ordering and backtracking 
algorithms to combine the solutions. If the resultant plan can be made correct, 
then one such correct plan is returned. Otherwise, it returns to the previous 
step to select the next set of plans for combination. The process repeats until 
either there is a successful combination, or there is no new combination to 
be considered. 

To test the efficiency of this strategy, experiments were done in the blocks 
world domain, where a planning problem is defined by the initial and final 
configurations of stacks of blocks on a table. The restrictions are that only 
one block can be moved at a time, and that a block cannot simultaneously 
support more than one block. The operators in this domain are move(x, y, 2:), 
for moving a block x from ^ to a block z, and newtower(x, y) for moving a 
block X from the top of y to the table. 

One way to decompose the blocks world domain is to consider the move- 
ment of each block as the task of a specialist. Suppose that a specialist knows 
exactly how a block x can be moved, in the following manner: From any ini- 
tial situation On(x, y) to a goal situation On(x, z), every block x is moved in 
precisely one of the following ways: 

1 . If y = z = Table, then return {(donothing)} as the only solution 

for moving block x. donothing denotes an empty sub-plan, in which 

all goal conditions are established by the initial situation. 

2 . If y = Table but 2: ^ Table, then return {(move(x, Table, z))}. 

3 . If y ^ Table and z = Table, then return {(newtower(x, y))}. 
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4. If y = z Table, then return a set of two alternative solutions: 
{(donothing), (newtower(x, y)^move{x, Table, z))}. 

5. Otherwise, return 

{(move(x, y, z)), (newtower(x, y)i-^move(x. Table, 2 ))}. 

The above domain-dependent enumeration of the movement of a block com- 
pletely characterizes its possible movement for any given initial and final 
situations. Therefore, given a blocks-world problem, if there is a plan for all 
blocks, then a sub-plan exists for each individual block that can be combined 
to result in a correct one. In other words, Watplan is complete for this do- 
main. The difficulty lies in the selection of sub-plans which can be combined 
to result in a final solution. When more than one block exists, a choice made 
may not only affect the successful movement of one block, but also make 
the other blocks’ movements either easier or harder, or even impossible. We 
illustrate the selection process through the following example. 

The initial situation is 

On(C,A),On(A,B),On(B,Table),Clear(C),Clear(Table) 
and the goal is 

On(A,B),On(B,C),On(C,Table). 

The initial sub-plans, which are provided by the specialists for the blocks, 
are listed below. 

For block A: {(donothing), (newt ower( A, B)H-^move( A, Table, B))} 

For block B: { (move (B, Table, C)} 

For block C: {(newtower(C, A))}. 

The first choice for sub-plan combination includes the sub-plan donothing 
for block A. However, when the three sub-plans are combined, no opera- 
tor can be found in the plan that establishes the precondition Clear (B) of 
move(B,Table,C). But the second choice for combination, listed below, can 
be successfully merged. 

For block A: (newtower(A, B)i-^move(A, Table, B)) 

For block B: move (B, Table, C) 

For block C : newtower(C, A) 

In particular, when the three sub-plans are combined, the newly- found 
causal links for precondition Clear(A) of move (A, Table, B) and precondition 
Clear (B) of mo ve(B, Table, C) require the imposition of ordering constraints 

newtower(C, A)i-»move(A, Table, B), newtower(A, B)i-^move(B, Table, C). 

Furthermore, the following conflicts are detected: 

1. move(A,Table,B) is a clobberer for the causal link 

^ move(B, Table^ C)) where p = Clear(B), and 
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Fig. 9.1. Comparison in blocks- world domain 



2. mo ve(B, Table, C) is a clobberer for the causal link 
(simt newtower((7, ^)) where q = Clear(C). 

A linear plan is obtained by resolving both conflicts: 

newtower(C,A) i— > newtower(A,B) mo ve(B, Table, C) move(A,Table,B). 

Empirical tests have been conducted with randomly generated blocks- 
world problems, which are simply randomly-generated initial situations for a 
given number of blocks. For each random problem, a domain-specific routine 
is first applied to generate the set of alternative movements for each block. 
Then Watplan is applied for selecting sub-solutions and resolving conflicts. 
To compare with an incremental planner, an additional run is made for each 
test problem using Tweak to combine sub-plans and resolve conflicts. The 
results are shown in Figure 9.1, where each datum is computed as the average 
result of 10 randomly generated problems for a given number of blocks. This 
test again demonstrates that with Watplan the computational cost of the 
combination phase is much lower than Tweak. 



9.2 Optimization-Based Plan Selection 

As with constraint-based multiple-goal plan-selection problems, it is often the 
case that more than one plan exists for each goal when the plans are merged. 
A computational difficulty occurs due to the need to choose one plan for each 
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goal so that after merging these plans, the total cost of the merged plan is 
minimized. 



9.2.1 Examples 

We start out by describing some example problem domains where the 

multiple-goal plan-selection problem is important. 

Example 1: Manufacturing Process Planning. Much work has been done in 
the area of process planning for manufacturing, where the aim is to de- 
velop a plan of action for producing a machined part. As an example, 
consider the problem of using machining operations to make holes in a 
metal block. Several different kinds of hole-creation operations are avail- 
able (twist-drilling, spade-drilling, gun-drilling, etc.), as well as several 
different kinds of hole-improvement operations (reaming, boring, grind- 
ing, etc.). Each time one switches to a different kind of operation or to a 
hole of a different diameter, one must mount a different cutting tool on 
the machine tool. If the same machining operation is to be performed on 
two different holes of the same diameter, then these two operations can 
be merged by omitting the task of changing the cutting tool. This and 
several other similar manufacturing problems are of practical significance 
(see [27, 63]). 

Example 2: Query Plan Optimization. Another important area of problems 
related to planning is that of multiple-query optimization in database 
systems. 

Let Q = {Qi, Q25 • • • ? Qn} be a set of queries to be processed. Associated 
with each query Qi is a set of alternate access plans {77^1, 77^2, - • • , 77^^.}. 
Each plan is a set of partially ordered tasks that produces the answer 
to Q. For example, one task might be to find all employees in some 
department whose ages are less than 30, and whose salaries are over 
$50,000. Each task has a cost, and the cost of a plan is the sum of the 
costs of its tasks. Two tasks can be merged if they are the same, or if 
the result of evaluating one task reduces the cost of evaluating the other. 
The multiple-query optimization problem [117] is to find a global access 
plan by selecting and merging the individual plans so that the cost of the 
global plan is minimized. 

Example 3: Common Sense Planning. Generating multiple plans may some- 
times lead to better results even if the least costly plan is generated first. 
Consider a planning situation described by Wilensky [145]: 

John lives one mile from a bakery and one mile from a dairy. The 
two stores are 1.5 miles apart. John has two goals: to buy bread 
and to buy milk. 

This time, however, let us add the fact (based on [145]) that 

John lives 1.25 miles from a large grocery store that sells both 
bread and milk. 
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The best plans for the individual goals involve two separate trips: one 
to the store and one to the dairy. However, this would require making 
two 2-mile trips for a total of 4 miles. The approach described in the 
previous section would allow them to be merged so that John could go 
directly from one store to the other (for a total trip of 1 + 1.5 H- 1 = 3.5 
miles). A better plan, however, is to use the second-best plan for each 
goal (going to the grocery store). Even though taken separately these 
would generate a worse plan (two 2.5-mile trips for a total of 5 miles), 
they permit more significant merging when combined together (a single 
trip of 1.25 -h 1.25 = 2.5 miles). Thus, if the planners for the individual 
trips delivered more than one solution for each goal, this better plan 
could be found. 

In all of the above domains more than one plan may be generated for 
each goal. This necessitates choosing among the plans available for each goal 
in order to find an optimal global plan. This problem in general is NP-hard, 
and in this chapter, we present a heuristic search technique that works quite 
well in practice. 



9.2.2 Complexity 

Our problem can be stated as follows. Given a set Q of goals gi^i = 1, 2, . . . fc, 
and given a set Vi of alternate plans for each goal set gi, select one plan for 
each goal such that the resulting merged plan has a minimal cost. 

This problem is NP-hard, even without considering the confiicts among 
the plans. We can prove this claim by a reduction from the hitting set prob- 
lem [55], which is known to be NP-complete. The hitting set problem can be 
stated as follows: 

Given a collection C of subsets of a finite set 5, and a positive integer 
K < 1 5 1, is there a subset S' C S with \S'\ < K such that S' contains 
at least one element from each subset in C? 

The transformation to a multiple-plan selection problem can be done as fol- 
lows. Every subset of 5 is a goal, and each element is a plan step. Two plan 
steps are mergeable if their corresponding elements are identical. 

Despite the fact that the problem is NP-hard to solve, polynomial-time 
solutions do exist for several special cases of the merged plan existence and 
optimal merged-plan problems. One special case is where the number of dif- 
ferent operator types is less than 3, assuming that plan steps of the same 
type can be merged. In this case, if no conflicting constraints are allowed to 
exist, the problem can be solved in polynomial time. For example, this would 
occur in Example 1 of Section 9.2.1 if there were only two different kinds of 
machining tools to be considered. 
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9.2.3 A Heuristic Algorithm for Plan Selection 

Since the multiple-goal plan-selection problem is NP-hard, we resort to a 
heuristic method. Our algorithm will be based on branch- and-bound^ intro- 
duced in Chapter 4. 

A Branch and Bound Algorithm. Suppose that for each goal gi we are 
given a set of plans Vi containing one or more plans for gi. Suppose also 
that we are given a number of operator types, such that plan steps of the 
same type can be merged. For a set of plans S = = 1,2,...}, we 

denote by COMBINE (5) the plan after all conflicts are resolved in the set, 
using constraint-based techniques developed in Chapter 7. We denote by 
Merge(5) the merged plan using either OptimalMerge or ApproxMerge 
algorithms of the last chapter. 

Our state space is a tree in which each state at the i-th level is a plan for 
the goals ^i, • • • , Pi- This tree is defined as follows (an example is shown 
in Figure 9.2): 

1. the initial state is the empty set; 

2. for each state S at level i of the tree, the children of S consist of all plans 
of the form Merge(Combine({5, U})) such that JJ € Vi^i is a plan for 
Si+l 

Every state at level A: is a final state, for these states are plans for the con- 
joined goal G = {gi, g 2 , . . . , gA:}- A goal state is a final state F for which 
Merge(Combine(F)) is minimal among all final states. 

To search the state space, we use the best-first branch-and-bound algo- 
rithm shown below (for an overview of branch and bound, see Chapter 4). 
This algorithm maintains an active (or open) list A that contains all states 




Fig. 9.2. An example state space. Here IJij is the j-th alternate plan for goal-set 
Gi 
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Table 9.2. The OptimalSelect algorithm 



Algorithm OPTIMALSELECT(iJ) 

Input: A set of plan sets Vi, i = 1, 2, . . . , A:, 

External Functions: 

Children{S) returns the successor states of S; 

L{S) is a lower bound function, and U{S) an upper bound function. 
Output: A merged global plan. 

1. A := {0}; 

I* (A is the branch- and-bound active set)^ j 

2. loop 

3. remove from A the state S for which L{S) is smallest; 

4. if 5 is a goal state then ret urn (5); 

5. else 

6. C := Children(S); 

7. for each state Si in A do 

8. for each state S 2 in C do 

9. if t/(5i) > L{S 2 ) then 

10. remove 5i from A; 

11. else if C/(52) > L{Si) then 

12. remove S 2 from (7; 

13. end if 

14. A:=AUC 

15. end if 

16. repeat 



eligible for expansion. To choose which member of A to expand next, the al- 
gorithm makes use of a function L{S) that returns a lower hound on the costs 
of all descendants of S that are goal states. It always chooses for expansion 
the state 5 € A for which L{S) is smallest.^ 

The algorithm OptimalSelect is shown in Table 9.2. 

Computing the Lower Bound. If L{S) is a true lower bound on the costs 
of all descendants of 5 that are goal states, then L is admissible, in the sense 
that Algorithm OptimalSelect will be guaranteed to return the optimal 
solution. Below we develop a lower bound function that is informed enough 
to reduce the search space dramatically in many cases. 

Let S be any state at level i in the state space, and T be any child of 5. 
Then T is the result of merging S with some plan U £ Vi+i. Let N{II,S) 
be the set of all steps in II that cannot he merged with steps in S. This is the 
set of steps in iJ that are of different operator types from any of those in S. 



^ The relationship between best-first branch and bound and the A* algorithm is 
well known [99]. The quantities L{S), Cost (5), and L{S) — Cost (5) used above 
are analogous to the quantities f{S), G(5), and h{S) used in A*. 
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We have 

Cost(T) > min Cost(iV(iJ, S)) + Cost(5). 

Uev.+i 

Similarly, if U is any child of T at level i + 2, then 



Cost(17) > Cost(5) + max 



min^jgp,^^ Cost(fV(77,5)), 
minj^gp,^^ Cost(iV(7T, 5)). 



By applying this argument repeatedly, we get 



Cost(V) > Cost(5) + max min Cost(N{II, S)) 
}>i IleVj 



(9.1) 



for every goal state V below S. Thus Li{S) = Cost(5) is an admissible 
lower bound function (this would correspond to using /i = 0 in the A* search 
algorithm). However, a better lower bound can be found from formula (9.1): 

L 2 {S) = Cost(5) + max min Cost(iV(i7, S)) 
j>i lleVj 



1/2 is an admissible lower bound function that is better than Li. 

Computing Cost (5). The lower bound computation depends on the cost 
of a state resulting from merging a set of plans. There are several ways to 
estimate the cost. 

1. Apply the dynamic programming algorithm OptimalMerge to the set 
of plans in state 5, thereby obtaining an optimal cost; or 

2. Apply the approximation algorithm ApproxMerge to the set of plans 
in state S, thereby obtaining a non-optimal cost; 

If we follow method 1 above, then obviously we could obtain an accurate 
measure on the cost of a merged plan in each state, and the resultant lower 
bound heuristics L\ and L 2 are accurate as well. 

Improving the Lower Bound. By making more intelligent use of the in- 
formation contained in the sets W(i7,5), we can compute an even better 
lower bound function. The function Ls{S) is described below. 

Let 5 be a state at level i in the state space, and let j, k be two indexes 
that are higher than i: j, k > i. We say that Vj and Vk are S -connected if 
either of the following conditions holds: 

- there are plans iJ G Vj and Q e Vk such that N{U, S) and iV(Q, S) 
contain some steps that are mergeable, or 

— there is a set Vh that is 5-connected to both II j and 77 k • 




152 9. Multiple-Goal Plan Selection 




V 



Fig. 9.3. Illustrating S-connected classes 



Class C1 



Class C2 



other classes 



5-connectedness is an equivalence relation, so we let Ci(5), (52(5), . . . , be 
the equivalence classes. Therefore each class Ck{S) contains one or more of 
the Vj^s. An example is shown in Figure 9.3. We refer to these equivalence 
classes as S -connectedness classes. 

Having done this, we can now define 

1 , 3 ( 5 ) = Cost(5) 4- ^ max min Cost(iV(iJ, 5)) (9.2) 

“ ^fcGCj(S) UeVk 

It can easily be shown that L 3 is a lower bound on the cost of any descendant 
of 5 that is a goal state. 

Computing an Upper Bound. The last factor in the heuristics of Algo- 
rithm OptimalSelect is the upper bound U{S) for each state 5. Compu- 
tation of the upperbound is ultimately related to whether one can quickly 
select one plan for each goal so that they can be consistently combined. This 
is a complex problem, as we have seen in Chapter 7. Without more domain 
knowledge, we cannot say much about whether we could succeed in finding a 
consistent set of plans to combine. However, in practical applications of the 
algorithm, as we see below in the manufacturing domain, strong knowledge 
is often available for guiding the selection process. 

For now, we will make the simplifying assumption that any selection of 
plans can be, in one way or another, combined into one in which all conflicts 
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are resolved. This assumption allows us to concentrate on the optimization 
aspect of the problem. 

Let 5 be a state at level i, and let {Vj^j = i 4- l,i 4- 2, . . . , A:} be the 
remaining plan sets. An upper bound estimate U (5) can be constructed by 
taking a minimal cost plan II j from each set Vj, and then computing the 
cost of a merged plan, using either OptimalMerge or ApproxMerge, of 
the plan set {5, ilj, j = i 4- 1, i 4- 2, . . . , /c}. This value is a true upper bound 
on the optimal solution under state S because it corresponds to the cost of 
a leaf node in the subtree under S. 

9.2.4 Examples 

We now demonstrate the application of Algorithm OptimalSelect in two 
domains. 

A Process Planning Example. Consider an example from a machining 
domain (see Example 2 of Section 9.2.1). Suppose that the goal is to drill two 
holes hi and /12 on a piece of metal stock. To make hole /ii, the plan is 

ill : spade-drill (/ii) bore(/ii). 

To make hole /i 2 , however, there are two alternative plans: 

II 21 : twist-drill(/i 2 ) ^ bore(/i 2 ). 

II 22 : spade-drill(/i 2 ) 1 — > bore(/i 2 ). 

Each time one switches to a different kind of machining operation, a different 
cutting tool must be mounted. Suppose that the relative costs are as follow: 
1 for each tool change, 1 for each twist drilling operation, 1 for each boring 
operation, and 1.5 for each spade drilling operation. Then the costs of the 
individual plans are Cost(i7i) = Cost(il 22 ) = 4.5 and Cost(il 2 i) = 4. 

At level 0, the initial state Sq = 0, and the plan sets V\ = {-^ 1 } and 
V 2 = {-^ 215 ^ 22 } are S'-connected since they share tool-changing operations 
for boring and spade-drilling. 

At level 1, the state-space has only one state, namely S\ = 
N{Il 2 i,Si) = {twist-drill(/i 2 )} and N{Il 22 ,Si) = 0. Thus, 

L^{Si) = Cost(5i) + min{Cost(iV(iJ2i,5i)),Cost(Ar(iJ22,5i))} = 4. 

At level 2, the two successor states of Si are: 

5 21 = {MERGE(COMBINE({i7i,il2i}))}, 

522 = {MERGE(COMBINE({iJi,i722}))} 

Their heuristic estimates are L^{S 2 i) = 7, 1 ^ 3 ( 522 ) = 6.5. Thus, the optimally 
merged plan is 

522 = spade-drill(/ii, / 12 ) i— > bore(/ii and /i 2 )- 
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A Household Domain. We now demonstrate Algorithm OptimalSelect 
on the “bread and milk” example, Example 3, from Section 9.2.1. A graphical 
illustration is shown in Figure 9.4. The plans for the goal have(Bread) are 

iJii: go (Home, Bakery) buy (Bread), go (Bakery, Home); 

^ 12 - go(Home, Grocery) buy(Bread), i— > go(Grocery, Home). 

The plans for the goal have(Milk) are 

7721 : go (Home, Dairy) buy (Milk) »-> go (Dairy, Home). 

-^ 22 - go (Home, Grocery) buy (Milk) go (Grocery, Home). 

We now trace the operation of Algorithm OptimalSelect. 

At level 1, there are two states, 

5i = {77n},52 = {7Ji2}. 

Taking the distance between any two locations as the cost of going from one 
to the other, we have 



Cost(5i) = 2,Cost(52) = 2.5. 

The heuristic function values are 

7s(5i) = 2 + min{Cost({buy(Milk)}), Cost(7722)} = 2 , and 
Lz{S 2 ) = 2.5 + min{Cost(772i),Cost({buy(Milk)})} = 2.5. 

Thus, the algorithm will expand ^i next. 

There are two successors of Si. 

Ti = {iJii,7l2i}; T 2 = {7Ii2,7l22}- 

The merged plan Ti corresponds to going to the dairy to buy milk, going 
from the dairy to the bakery to buy bread, and finally going home from the 



John 




Fig. 9.4. A shopping example 
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bakery. The cost of this plan is Cost(Ti) = 1 + 1. 5 + 1 = 3.5. T 2 corresponds 
to going to the bakery and the grocery store on separate trips, giving rise to 
a cost of 4.5. Since both are more costly than 52 , S 2 will be expanded next. 
One successor of S 2 combines and merges II 12 with II 221 yielding the plan 

go (Home, Grocery), buy (Milk and Bread), 1 -^ go (Grocery, Home). 

In this case, this plan is the optimal merged plan, with a cost of 2.5. 

9.2.5 Empirical Results in a Manufacturing Planning Domain 

To test the effectiveness of the heuristics, experiments are done by Nau and 
Yang with the algorithm using the EFHA process planning system [136], a 
domain-dependent planner based on the earlier SIPS process planner, devel- 
oped by Nau et al. at University of Maryland [98]. The parameters involved 
in the generation of alternate plans are varied to make sure they would not be 
overly uniform. Particular attention are paid to ensure that the test problems 
are typical of those found in manufacturing applications. 

The target problem is to find a least-cost plan for making several holes in 
a piece of metal stock. Specifications for 100 machined holes are generated, 
randomly varying various hole characteristics such as depth, diameter, surface 
finish, locational tolerance, etc. EFHA is executed to produce at most 3 plans 
for each hole, and it found plans for 81 of the holes (for the other 19 the 
machining requirements were so stringent that EFHA could not produce any 
plans using the machining techniques in its knowledge base). 

The distributions of the hole characteristics are chosen so that the plans 
generated for the holes would have the following characteristics: 

1 . a wide selection of plans, rather than lots of duplicate plans for different 
holes; 

2 . not very many holes having an “obviously best” plan (i.e., a plan that is 
a sub-plan of all the other plans for that hole); 

3. lots of opportunities to merge steps in different plans; 

4. a large number of “mergeability tradeoffs” in choosing which plan to use 
for a goal. That is, it is nontrivial to decide which alternative plans to 
select in order to minimize their costs. 

The results of the experiments are shown in Table 9.3. Each entry in the 
table represents an average result over 450 trials. Each trial was generated 
by randomly choosing m of the 81 holes (duplicate choices were allowed), 
invoking Algorithm OptimalSelect on the plans for these holes using the 
lower bounding function L 3 , and recording how many nodes it expanded in 
the search space. The total cost of each plan was taken to be the sum of the 
costs of the machining operations in the plan and the costs for changing tools. 

The performance of the algorithm are quite good — especially since the 
test problem was chosen to be significantly more difficult than the kind of 
problem that would arise in real-world process planning. In real designs, 
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Table 9.3. Experimental results for Algorithm 4 using Ls 



Number of holes m 


Nodes in the search space 


Nodes expanded 


1 


2 


1 


2 


10 


2 


3 


34 


3 


4 


98 


4 


5 


284 


6 


6 


852 


9 


7 


2372 


12 


8 


6620 


16 


9 


19480 


22 


10 


54679 


28 


11 


153467 


38 


12 


437460 


51 


13 


1268443 


61 


14 


3555297 


86 


15 


9655279 


110 


16 


29600354 


170 


17 


80748443 


223 


18 


250592571 


250 



designers would normally specify holes in a much more regular manner than 
our random choice of holes, making the merging task easier. For example, 
when merging real-world process plans, many of the mergeability tradeoffs 
mentioned earlier would not actually occur; and without such tradeoflFs, the 
complexity of the algorithm is polynomial rather than exponential. 



9.3 Open Problems 

Combining OptimalS ELECT with Watplan. One problem which we did 
not address in this chapter is how to extend the merging-based plan-selection 
technique to include constraint-based plan selection in Watplan. Watplan 
can be seen as a satisficing method, for selecting a plan for each goal set 
such that a consistent combination can be obtained. OptimalSelect, on 
the other hand, is an optimization algorithm for selecting plans based on 
quality. To combine the two algorithms, one could use plan merging as a 
basis for designing a value-selection heuristic in Watplan. This heuristic 
prefers a low-cost plan whenever choices are available. The resultant solution 
found in the end may not be the one with absolutely the lowest cost, but it 
will be a good one. 

Interleaving Plan Selection with Plan Generation. The problem model 
we have so far relied on assumes that a set of alternative plans is given for 
each goal-set. It is possible that to generate the alternatives, one has to 
spend vast computational resources. Therefore, it would be more efficient if 
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one could select plans as they are generated, rather than waiting till all plans 
are constructed. We suspect that to do this one needs to have some special 
kind of knowledge about the problem domain, so that even before a plan is 
completely generated, some prediction could be made regarding whether the 
plan can be fruitfully combined and merged with others. 



9.4 Summary 

The multiple-goal plan-selection problem is NP-hard. Constraint-based as 
well as a branch-and-bound heuristic-search algorithms are developed to find 
conjoined plans. An admissible heuristic, and several variants, are proposed 
for the merging-based multiple-goal plan-selection problem to show that this 
search can find optimal plans. Empirical results are shown demonstrating 
that in an interesting class of automated manufacturing problems, the heuris- 
tic algorithm performs quite well, still growing exponentially but by a very 
small factor. 



9.5 Background 

Discussion on consistency-based multiple-goal plan selection problem was 
partly based on an Artificial Intelligence journal article by Yang [154]. Discus- 
sion on merging-based multiple-goal plan selection problem is adapted from 
Yang’s PhD dissertation [150], part of which was reported in a Computa- 
tional Intelligence journal article by Yang, Nau, and Hendler [158]. Ephrati 
and Rosenschein presented an integrated framework for distributed planning, 
which could handle both conflicts and operator-overlapping interactions while 
a global plan is being generated [42] . A complete description of the application 
of merging-based multiple-goal plan selection methods to process planning is 
described in [76]. 




Part III 



Hierarchical Abstraction 




Overview 



A superior strategist develops a plan 
Before committing the entire force. 

An inferior strategist commits the entire force 
Before developing a plan 

(Sun Tzu, The Art of Strategy, Chapter Four) 



In this last part of the book we focus on hierarchical planning with ab- 
straction. Planning with abstraction refers to the methodology in which an 
abstract version of a plan is first developed by concentrating on the most 
important aspects of a problem. After committing to the abstract plan, a 
refinement process will then follow to complete all the remaining details. 

Planning with abstraction has had a long tradition in Artificial Intel- 
ligence, but the development of this subject into a vigorous discipline has 
been rather recent. In Chapter 10 we give a general introduction to abstrac- 
tion, providing a basic computational framework in which we could design 
algorithms and perform analysis. In this chapter we also describe an array of 
properties with which we could judge objectively the quality of an abstraction 
hierarchy. With the formalism at hand, we then turn around and develop an 
algorithm for automatically generating abstraction hierarchies (Chapter 11). 
This work is built on previous achievements in the field, which we review in 
detail in this chapter. In Chapter 12 we examine another type of hierarchi- 
cal planner, using hierarchical task networks. We show that some important 
formal properties are lost with this type of planner, and we also illustrate 
how to remedy these losses. Finally, in Chapter 13, we present a method for 
automatically abstracting the effects of planning operators. The method is 
based on a learning algorithm, which is able to find an appropriate balance 
between the completeness of a planner and its computational efficiency. 




10. Hierarchical Planning 



Ever since the conception of Artificial Intelligence, hierarchical problem solv- 
ing has been used as a method to reduce the computational cost of planning. 
The idea of hierarchical problem-solving, a well-accepted one, is to distin- 
guish between goals and actions of different degrees of importance, and solve 
the most important problems first. Its main advantage derives from the fact 
that by emphasizing certain activities while temporarily ignoring others, it 
is possible to obtain a much smaller search space in which to find a plan. 

As an example, suppose that in the household domain we would like to 
paint the ceiling white. Initially the number of conditions to consider may be 
overwhelming, ranging from the availability of various supplies, the suppliers 
for equipments and tools, to the position of the agent, the ladder, and the 
state of the ceiling. However, we could obtain a more manageable search 
space by first concentrating on whether we have the paint, the ladder, and 
a brush. Once a plan is found we then consider how to refine this plan by 
considering how to get to the rooms where each item is located. The process 
repeats until a full-blown plan is finally found. 

Today, a large number of problem-solving systems have been implemented 
and studied based on the concept of hierarchical planning. They include GPS 
[ 101 ], Abstrips [ 113 ], Lawly [ 121 ], Noah [ 114 ], Nonlin [ 132 ], Molgen 
[ 128 ], Soar [ 138 ], Sipe [ 146 ], and AbTweak [ 160 ]. 

In this part of the book, we provide a general introduction of hierarchical 
planning, exploring the uses of hierarchies in planning in several contexts. 
We first provide a generic introduction to hierarchical planning, in which a 
plan is gradually refined by considering increasing levels of detail. We then 
specifically consider two ways in which details are inserted in a plan. The first, 
precondition- elimination abstraction^ mimics the human intuition of exploring 
and solving subgoals in order of importance, with the most important goals 
solved first. 

A second method of hierarchical planning that we consider is called hierar- 
chical task-network planning (HTN), where a planning problem and operators 
are organized into a set of tasks. A high-level task can be reduced to a set 
of ordered lower-level tasks, and a task can be reduced in several ways. The 
reduction functions as a mapping from a task to a set of subtasks, defining 
the knowledge about how one can obtain a detailed plan from an abstract 




164 10. Hierarchical Planning 

one. These two methods are complementary — each can be very effective in 
solving a different type of problem. 

10.1 A Hierarchical Planner 

10.1.1 Algorithm 

The intuition behind the operation of a hierarchical planner is shown in 
Figure 10.1. In this figure there are three levels of abstraction, an abstract 
level, an intermediate level and a concrete level. Each dashed box represents 
a problem-solver at a given level. A planning problem is first abstracted and 
solved at the most abstract level. The solution obtained at this level, an 
abstract plan^ is taken as the input to a problem-solver at the next level. The 
process ends when a concrete-level solution is found. 

In general, the abstraction levels could range from a single level to multiple 
levels. The former is identical to problem-solving without any abstraction. 
A top-level description of a hierarchical partial-order planner is shown in 
Table 10.1. 

For a given problem. Algorithm HiPlan starts with an initial plan Uq 
representing the start-step/finish-step pair. It then iterates through a cycle 

Abstract 
Level 



Intermediate 

Level 



Concrete 

Level 




Abstract Plan 
I 




I 



T 

Refined Plan 



i 




Concrete-level Plan 



Fig. 10.1. Illustrating hierarchical planning 
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Table 10.1. The HiPlan algorithm 



Algorithm HiPlan(77o) 

Input: An initial abstract plan /Jo- 

External Function: Refine(/7, i) returns a set of refinements of plan at abstraction 
level i. 

Output: A correct plan consisting of concrete-level plan steps; or Fail. 

1. OpenList := {/Jo}; 

2. repeat 

3. 77 Remove(OpenList); 

4. if Level (/7) = 0 then 

5. if Correct (/J) then 

6. return(II); 

7. end if 

8. else 

9. Level(IT) := Level(/7) - 1; 

10. Successors := Refine(/7, Level(II)); 

11. OpenList := Insert(Successors, OpenList); 

12. end if 
ISuntil OpenList = 0; 

14return(Fail); 



of refinement^ where in each iteration, a plan is chosen from the OpenList 
(Step 3), and if it is not a correct concrete-level plan, it will be lowered 
to the next level down (Step 9). The plan at this new level is likely to be 
temporarily incorrect, since some plan steps may be left out. To account for 
these missing components a refinement process (Step 10) will be initiated, in 
which more plan steps are filled in. The algorithm stops with a correct plan 
when it finds a plan II consisting solely of concrete-level plan steps, and is 
correct (Steps 5,6). 

In the algorithm, the function Refine(/7, i) is intentionally left unspecified. 
Depending on the type of hierarchical planning used, we could specify this 
function either explicitly or implicitly. In the next section, we conduct a case 
study where this function is specified implicitly. 



10.1.2 Precondition-Elimination Abstraction 

Precondition-elimination abstraction systems depend on a formulation of a 
problem domain as a multi-layered hierarchy. The form of abstraction hier- 
archy largely determines the eflFectiveness of problem-solving, and not every 
hierarchy will lead to an improvement in eflficiency. 

In Abstrips [113], Sacerdoti developed an elegant means for spedf}dng 
abstract problem descriptions by assigning criticality values. Here we follow 
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{at(l2)} j 



Abstracting Operators 



' Preconditions: 

[ (at(l1), have(car)} 


i 


' Preconditions: ' 

j {at(l1), have(truck)} , 


I operator 


drive-car(l1, 12) 




I operator 


drive-truck(l1 , 12) i 


I Effects: 


{at(l2)} 


I 

I 
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I 
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V 


I 

{at(l2)} \ 



Criticality 


Preconditions 


1 


at(l1) 


0 


at(l1), have(truck), 
have(car) 



Fig. 10.2. Two operators with the same abstract representation 



the same approach, but do not restrict our planning algorithm to a total-order 
planner such as Abstrips. 

Defining a Hierarchy. Consider assigning the integers between 0 and n— 1, 
for some finite value n, to preconditions. A problem description at level- 
i is obtained by eliminating all literals having criticality less than i from 
the precondition of an operator. This procedure defines an n- level planning 
system — a triple U = {L,0^ crit), where L is the set of literals used in 
describing the problem domain, and O is the set of operators, crit is a function 
mapping preconditions to non- negative integers: 

crit : y Pre(a) ^ {0, 1,.. ,,n - 1}. 
aeo 

Level 0 is called the concrete level, and level n — 1 the most abstract level. 
The level of a plan is denoted as Level (iJ). 

Intuitively, crit is an assignment of criticality values to each literal ap- 
pearing in the precondition of an operator. Let a be an operator. We take 
Abs(i, Pre(a)) to be the set of preconditions of a which have criticality values 
of at least i: 



Abs(i, Pre(a)) = {p \ p ^ Pre(a) and crit{p) > i}. 

Abs(i,a) is operator a with preconditions Abs(i, Pre(a)) and effects Eff(a). 

Any abstract operator a may be the result of abstracting more than one 
lower- level operator. Figure 10.2 shows an example. Let ai, a 2 , . . . , all 
have the same abstract operator representation at level i, then the cost of the 
abstract operator is the minimum of the costs of all operators i = 1, 2 ... A:. 
For every abstract operator a at level i, we use NextLevel(i, a) to denote 
the set of i — 1 level operators, such that every operator in the set can be 
abstracted to a at level i. 

Finally, let the set of all such Abs(i,o:) be Abs(i,(9). This defines a 
planning problem description on each level i of abstraction. The domain- 
description language and the planning operators are specified as: 
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(L, Abs{i,0)) 

The same definition applies to plan steps; for a step s in a plan at the concrete 
level, level 0, the preconditions of an abstract step at the i-th level are given 
by 

Abs(i,Pre(s)) = {p \ p £ Pre(s) and crit{p) > i} 

The effects of Abs(i,s) remain the same, and the cost of Abs(i,s) is defined 
the same as the cost of an abstract operator. 

In our definition of an abstraction hierarchy we have restricted our at- 
tention only to the preconditions. This type of abstraction has been called 
relaxed models^ since by ignoring certain preconditions, a planning problem 
is simplified, and the number of solutions to a planning problem increases. 
If one further abstracts the effects of operators then in fact one can obtain a 
much smaller search space for a forward-chaining planner, and the resultant 
model is called reduced model. Knoblock [80] presents an excellent discussion 
of the two models, his conclusion being that most formal results obtained in 
one model are applicable to the other. 

A Tower of Hanoi Example. We illustrate the specification of an abstrac- 
tion hierarchy in the 3-disk version of the Tower of Hanoi (TOH) domain. 
As shown in Figure 10.3, initially all three disks are on Pegl. Disks can be 
moved one at a time, under the restriction that no disk of a larger size can 
rest on a smaller disk. The goal is to place all three disks on Peg3. 

We can represent the domain with four predicates, Ispeg, Onsmall, 
Onmed, and Onlarge. The concrete-level operators in this domain are shown 
in Table 10.2. If a hierarchy is built by assigning a distinct criticality value 
to each of the predicates, then 24 different hierarchies exist. We show one 
criticality assignment in Table 10.3. This assignment has often been used to 
demonstrate the advantages of abstraction, most notably by Knoblock [80]. 
For ease of exposition, we use ILMS to represent the hierarchy. 

The initial state of this problem is 

Ispeg(Pegl), Ispeg(Peg2), Ispeg(Peg3), 

Onlarge(Pegl), -iOnlarge(Peg2), -iOnlarge(Peg3), 
Onmed(Pegl), -i Onmed (Peg2), -> Onmed (Peg3), 
Onsmall(Pegl), ->Onsmall(Peg2), -»Onsmall(Peg3) 

Initial State; Goal State: 




Peg1 Peg2 Peg3 



Peg1 Peg2 Peg3 



Fig. 10.3. A Towers of Hanoi domain 




168 10. Hierarchical Planning 

Table 10.2. Operator definitions in the TOH domain 



Preconditions 


Effects 


Cost 


movelaxge(?x, ly) 


Ispeg(?x), Ispeg(?^), 

Onlarge(?x), 

-iOnmed(?x), 

-iOnmed(?y), 

-iOnsmall(?a:), 

-iOnsmall(?y) 


Onlarge(?y), -iOnlarge(?x) 


1.0 


movemed(?x, ly) 


Ispeg(?x), Ispeg(?7/), 

Onmed(?x), 

-iOnsmall(?rr), 

-iOnsmall(?t/) 


Onmed(?y), -iOnmed(?a:) 


1.0 


movesmall(?x, ly) 


Ispeg(?x), Ispeg(?i/), 


Onsmall(?y), 


1.0 


Onsmall(?x) 


-iOnsmall(?a:) 





Table 10.3. A criticality assignment in the TOH domain 



Criticality 


Predicates 


2 


Ispeg, enlarge 


1 


Onmed 


0 


Onsmall 



And a goal is 



Onlarge(Peg3), Onmed(Peg3), Onsmall(Peg3) 

For the ILMS hierarchy, at the most abstract level, the goal is 

Onlarge(Peg3) 

And the level-2 abstract operators are shown in Table 10.4. 

The TOH domain is of interest in the study of abstraction because of a 
number of problem features. A planning problem in this domain typically con- 
sists of conjunctive goals, and the solution length increases exponentially with 
the number of goals. Many non-hierarchical planners consider this problem 
hard to solve. In addition, there exists a clear distinction of different degrees 
of importance among the domain conditions, and the interactions between 
these conditions can be carefully controlled to generate meaningful results, 
as we will see later. 
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Table 10.4. Abstract operator definitions in TOH domain 



Preconditions 


Effects 


Cost 


Abs(2, movelarge(?x, ??/)) 






Ispeg(?a;), Ispeg(??/), 

Onlarge(?x) 


Onlarge(?y), -iOnlarge(?a:) 


1.0 


Abs(2, movemed(?x, ?j/)) 
Ispeg(?x), Ispeg(? 2 /) 


Onmed(?t/), -iOnmed(?x) 


1.0 


Abs(2, movesmall(?x, ? 2 /)) 
Ispeg(?x), Ispeg(?y) 


Onsmall(? 2 /), 

-iOnsmall(?x) 


1.0 



A solution at the most abstract level - level-2 of the ILMS hierarchy - is 
Hi = Si„itt-^movelarge(Pegl,Peg3)i-^s/j„isft 
At this level there are also other more costly abstract solutions. One of them is 
II2 = Si„itH^.movelarge(Pegl,Peg2)i-^-movelarge(Peg2,Peg3)H-»s/i„ish 
An optimal solution with seven steps can be found by refining U i : 

^init 

i 

moves (Pegl, Peg3) 

i 

moveM(Pegl, Peg2) 

i 

moves (Peg3, Peg2) 

i 

moveL(Pegl, Peg3) 

i 

moves (Peg2, Pegl) 

i 

moveM(Peg2, Peg3) 

i 

moveS(Pegl, Peg3) 

i 

^finish 

A Simple Travel Domain. We next consider a simple travel domain, where 
the task is to determine a route for driving from one city to the next (see 
Figure 10.4). To go to a destination city, it may be necessary to stop over 
in a number of intermediate cities, and within each of these cities, it may be 
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City A CityB 




necessary to find a route to connect the entry point to an exit point. The 
operators of this domain are listed in Table 10.5. 

An initial state of a planning problem can be specified as 

In-City(A), At-Loc(Exit^), Connected (Exit A, Entry^, B), 
Connected(ExitB, Entry C), . . . , Connected(Exitx, Entryy , F). 

A goal might be 

In-City(F) 

To solve this problem, an abstract plan corresponds to an inter-city route, 
and a concrete-level plan corresponds to one that has all intra-city routes 
planned out. 

Table 10.5. Operators in a simple travel domain 



Preconditions 


Effects 


Cost 


drive~bet-cities(?ci, IC 2 ) 

Connected (?extt, 7ci,?entry, ?C 2 ), In-City(?C 2 ), At-Loc(?ent'ry), 


10 0 


In-City(?ci), At-Loc{? exit) 
drive-in“city(?c, ?/i, 7 / 2 ) 
In-City (?c), At-Loc(?/i) 


-iIn-City(?ci), -^At-Loc(? exit) 
At-Loc(?/ 2 ), -'At-Loc(?Zi) 


1.0 
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10.2 Specifying Refinement 

As with Abstrips, HiPlan performs planning level by level, in a top-down 
fashion. Once a plan is found at a given abstraction level i, all preconditions at 
level i — 1 — the immediate lower-level — are put back into the preconditions 
of plan steps. As a result of the new preconditions, the plan may not be 
correct at level i — 1 . This calls for a process to refine the plan, to make 
it correct again at level i — 1. Below, we discuss the refinement process in 
two settings, with forward-chaining, total-order planners and with backward- 
chaining, partial-order planners. 

10.2.1 Forward- Chaining, Total- Order Refinement 

A total-order plan iJ at abstract level i can be considered as a skeletal plan 
for the next level down. Let an i-th-level plan iJ be a sequence of plan steps 

So = init i-^sih^S 2 ^“> . . . goal = s,. 

At the next lower level i — 1 , between every pair of steps (sj, 8 ^ 4 . 1 ) there 
appears a “gap” problem due to the introduction of new preconditions at 
level z — 1 . In the simple travel example introduced in the last section, an 
abstract-level plan consists of inter-city driving specifications only. When 
taken down to the intra-city level, however, between every “entry” point to 
a city and an “exit” point from the same city, a route has yet to be specified. 
The route is the gap problem. Figure 10.5 shows an example. 

Each of the gap problems is called a subprohlem. Before we can call a 
problem-solver for solving the problem, we must fully specify its initial and 
goal states. The goals for the subproblems are easy to specify; they simply 
consist of all preconditions of the next step. The initial states, however, might 
depend on the plans found for all gap subproblems before the current step. 

To address this issue, we follow the following length-first method of refine- 
ment [113]. The subproblems can be solved in a left-to-right scanning oper- 
ation. Starting from the left, we number the subproblems from 1 through r 
(see Figure 10.5). The initial state of the first subproblem is simply the initial 



(step 0) (step r) 




Fig. 10.5. Refinement of a total-order plan 
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state of the concrete problem. The goal is the set of all preconditions of the 
first plan step, si, at level i — 

goali — Pre(Abs(i — 1, Si)) 

A solution to this problem is obtained by a call to a forward-chaining, total- 
order planner II i = ToPLAN(mi^, goali). For a specification of ToPlan see 
Chapter 2. 

To obtain the initial state for the next subproblem, we apply the plan 
to the initial state, and obtain a new initial state mz^ 2 , for the next 
subproblem. In the j-th iteration, suppose that all previous j — I subprob- 
lems have already been solved. Let the solutions be II mi = 1, 2, . . . j — 1, 
respectively. Then for the j-th subproblem, the initial state is obtained by 
applying the plan 

II — ^Il2^ — ^^21 — ^IIs ■ • • Sj_x 

to the original initial state. This process continues until the final goal is 
achieved. 

A formulation of the above length- first refinement algorithm Total- 
Refine() is shown in Table 10.6. In this algorithm, 0^ is the set of concrete- 
level operators. Applyplan is a function that applies a plan to a state, and 
returns the resulting state. 

10.2.2 Backward-Chaining, Partial-Order Refinement 

For partial-order refinements, there is a higher degree of continuity between 
planning at one level and planning at the next. For any given plan iJ at 
an abstraction level i, all plan steps are first replaced by their correspond- 
ing (i — l)-th-level steps. If an abstract step s has several corresponding 
{i — l)-th-level steps, as specified in set NextLevel(i — l,s), one of the steps 
is chosen and the rest saved as backtrack points. Subsequently, the new pre- 
conditions will be planned for at level i — 1. The algorithm Parti alRefine 
is shown in Table 10.7. 

The continuity between planning at different levels of abstraction is a 
direct consequence of the flexibility offered by partial-order plans. Because the 
order in which a subgoal is achieved is not committed to until necessary, given 
two subgoals, a partial-order planner can derive the same plan by working 
on either subgoal first. In abstraction, this means that the rigid, left-to-right, 
length-first refinement of a total-order plan can be relaxed dramatically by 
allowing any open-precondition to be worked on without sacrificing the ability 
to find a refinement. In this way, an abstract partial-order planner works in 
the same way as a single-level planner, with abstraction providing a more 
intelligent ordering in which to achieve its subgoals. 
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Table 10.6. The TotalRefine algorithm for precondition-eHmination abstraction 



Algorithm TotalRefine(/7) 

Input: A plan 11 at abstraction level i. 

Output; A plan at abstraction level i — 1, or Fail. 

1 . i := crit{n)\ 

2. for each step s G Steps (77) do 

3. replace s by a step in NextLevel(i — 1, s); 

/* if there are several steps in NextLevel(z — l,s), 
pick one, and keep the rest as backtrack points. */ 

4. end for 

5. let init be the initial state in 77; 

6. let O be Abs(i — 1, Ob); 

/* Ob are the concrete-level operators */ 

7. let 77 consist of r + 1 steps (initial state is step 0) ; 

8.. for j = 1 to r do 

9. llj := ToPLAN(mz7 Pre(Abs(i — l,Sj))); 

/* if there are several solutions, pick one, 

and keep the rest as backtrack points. */ 

10. if 77j = Fail then 

11. return(Fail); 

12. else 

13. init := ApPLYPLAN(77j, mi^); 

14. end if 

15. end for 

16. Refinement Sjnit* — ^III^ — ^■si . . . 77r* — finish; 

17. return(REFlNEMENT); 



Table 10.7. The PartialRefine algorithm for precondition-elimination abstrac- 
tion 



Algorithm PartialRefine(77) 

Input: A plan 77 at abstraction level i. 

Output: A plan at abstraction level i — 1, or Fail. 

1. i := crit{n); 

2. for each step s G Steps(77) do 

3. replace s by a lower- level step in NextLevel(z — 1, a); 

/* if there are several steps in NextLevel(i — l,s), 

keep the rest as backtrack points. */ 

4. end for 

5. let O be Abs(z — 1, Ob); 

6. Solution := Poplan(77); 

/* for a description of the POPLAN algorithm see Chapter 2 */ 

7. return(Solution); 
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10.3 Properties of an Abstraction Hierarchy 

Given a problem domain represented by a description language L, a set 
of concrete-level operators and initial and goal states, there are many 
possible abstraction hierarchies that one can construct. With precondition- 
elimination hierarchies, if L contains N literals, and if we place one literal 
on each level of the abstraction hierarchy, we could obtain a total of N\ ab- 
straction hierarchies. Out of this many candidates, which one is good? This 
question has intrigued many planning researchers. 

One way to answer the above question is to define a property of an ab- 
straction hierarchy in terms of the relationship between an abstract plan and 
a concrete- level plan. The properties could presumably be linked to efficiency- 
gains in planning using the hierarchies. If a hierarchy satisfies a certain prop- 
erty, then we could classify it as a “good” one. This is the approach we will 
follow in this section. 

10.3.1 Existence-Based Properties 

Does the existence of an abstract solution guarantee the existence of a solu- 
tion at any lower level? Or, is the converse true of an abstraction hierarchy? 
In this section, we first review two properties on abstraction hierarchies based 
on existence of plans. 

Upward Solution Property. Consider a two-level hierarchy. Suppose that, 
whenever there exists any concrete- level plan ilc, there exists an abstract plan 
ila- Then the hierarchy is said to satisfy the upward solution property. 

In general, suppose that we are given an n- level abstraction hierarchy, 
with level n — 1 being the most abstract and level 0 the most concrete level. 
Let i be an integer between 0 and n — 2. 

Definition 10.3.1 (Upward Solution Property). Whenever any i-th- 
level solution II i exists, there exists an abstract solution Ili^i at level i 4- 1. 

By this definition, if a concrete-level solution II q exists then there exists a 
sequence of abstract solutions ending with iJo, (iln-i, • • ■ , such that 
each Hi is an i-th level abstract solution. Figure 10.6 shows a schematic 
diagram of this property. 

By the upward-solution property, if there is no solution plan for a problem 
at an abstract level, then there is no solution at any lower level either. This 
fact follows directly from the upward-solution property, since, if otherwise, 
a solution at any lower level would imply that solutions should exist at all 
higher levels, contradicting our initial assumption. This implication of the 
upward-solution property justifies our use of a top-down refinement strategy 
when planning with an abstraction hierarchy. 
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Fig. 10.6. Upward-solution property; in the figure every node marked sol is a 
solution node, and those marked with X are non-solution nodes. 



Downward Solution Property. Now consider the existence property in 
a top-down direction. Suppose that, for a two- level hierarchy, an abstract 
solution exists. If this entails that a concrete- level solution also exists, then 
we say that the hierarchy satisfies the downward solution property. 

In general, for an n-level abstraction hierarchy, we have the following 
definition. Let i be an integer ranging from 1 to n — 1. 

Definition 10.3.2 (Downward Solution Property). Whenever any i-th- 
level abstract solution II i exists, there exists an {i — l)-th-level solution 

By this definition, if an abstract solution Iln-i exists, then there exists a 
sequence of abstract solutions ending with ilo, {iJn-i, • • • , ilo}, such that 
each Ili is an i-th level solution. 

By the downward solution property, if any solution is found at an abstract 
level, then we know immediately that a concrete-level solution exists for the 
original planning problem. Conversely, if there is no solution at the concrete 
level, then no solutions exist at any higher level either. 

10.3.2 Refinement-Based Properties 

In the definition of the above properties, one feature that is missing is the 
specification of a precise relationship between an abstract solution and a 
lower level solution. With upward-solution property, a concrete- level solution 
lie implies the existence of an abstract solution But how is ilc related 
to 77a? 

In this section, we specify a precise correspondence between solutions us- 
ing a refinement relationship. By restricting the kind of refinement relations 
allowed, we could turn the above properties into useful search-control heuris- 
tics. In AI, transformations of this kind, where some abstract properties are 
turned into usable search control rules, are often referred to as operational- 
ization. 
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Monotonic Property. Recall from the previous section that, with a precon- 
dition-elimination hierarchy, an abstract plan could be “refined” to a lower- 
level plan using the algorithms TotalRefine or PartialRefine. Each al- 
gorithm is capable of returning several alternative refined plans for any given 
abstract plan. In general we can assume that there is a one- to- many relation 
Refine which, for any given abstract solution II at level i, returns a set of 
solutions {ill, 172, • • • ? Rt level i — \. 

Assume further that each plan Ili in Refine(il) leaves all causal links of 
n intact. During the transition from II to II no plan steps and causal links 
are removed from II. Then Ili is called a monotonic refinement of il. In 
TotalRefine and PartialRefine algorithms for precondition-elimination 
hierarchies, a monotonic refinement can be obtained by requiring that, when 
a new step is added to a plan, all threats with respect to a causal link that 
has been established at an abstract level are resolved. With a monotonic 
refinement, a plan never decreases in size as it evolves from one level to the 
next. 

With the notion of monotonic refinement, we can now specify a variant 
of the upward-solution property. Let i be an integer between 0 and n — 2. 

Definition 10.3.3 (Monotonic Property). Whenever an i-th-level solu- 
tion Ui exists ; there exists an abstract solution iJi-fi at level i + 1, such that 
Ili is a monotonic refinement of II i^i. In other words, Ili G Refine(i7i4_i). 

By this definition, if a concrete-level solution IIq exists then there exists 
a sequence of abstract solutions ending with ilo, {iJn-i, • • • , TIq}, such that 
each Ili is an i-th level abstract solution, and every plan ili_i is a monotonic 
refinement of a previous plan Ili. 

If the monotonic property is established for an abstraction hierarchy, we 
could then use the following search heuristic during plan generation, with 
either a total-order planner or a partial-order planner: during refinement, 
if any abstract- level causal link is threatened, and if the threats cannot be 
resolved, then the plan can be discarded from the search space. This practice 
will not sacrifice any possible solutions. 

With precondition-elimination hierarchies, upward-solution property al- 
ways holds, implying that the monotonic property holds too. This result, 
first proven by Tenenberg [134] and then refined by Knoblock [80], can be 
derived by the following argument. Let 77c be a concrete-level solution. We 
can replace each step in 77c by its corresponding abstract-level step, simply 
by removing all level-0 preconditions. The resulting plan 77' is still a correct 
solution at the abstract level. Now, this plan is likely to contain a few re- 
dundant steps — steps that are not useful in achieving the final goal. If we 
remove these steps and their associated causal links, while maintaining the 
correctness of the plan, we could obtain a plan with three properties: 

— the plan is still correct at the abstract level; 

— the plan can be monotonically refined to 77c; and 
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— the plan can be obtained by a planner, total-order or partial-order, at the 
abstract level. 

By repeating this procedure for all higher levels, we could eventually obtain 
a sequence of plans, such that every plan in this sequence is a monotonic 
refinement of the previous plan. 

Ordered Monotonic Property. With precondition-elimination hierarchies, 
the monotonic property is a universal property, in the sense that it is satisfied 
in every precondition-elimination hierarchy. When a property is universal, it 
is very likely that it has only weak heuristic power as a result of the well- 
known generality-effectiveness tradeoff. This is the case with the monotonic 
property; the monotonic property was implemented in an abstract planning 
system AbTweak [ 160]. It was found that in many cases, excessive back- 
tracking is incurred due to violations of the abstract causal links. The end 
result is that for many hierarchies there is no observable improvement in 
planning efficiency over not using abstraction at all. 

Similar observations led Knoblock to believe that it might be necessary 
to do away with backtracking across abstraction levels all together. This is 
enforced by a stronger property, known as the ordered monotonic property. 
Before explaining this property, we have to define what it means by an ordered 
refinement. A plan Hi is an ordered refinement of an abstract plan iJ if 

— is a refinement of iJ, and 

— the new plan steps added into the abstract plan II do not add or delete 
any literals with a higher criticality value. 

In the simple-travel example, suppose that all city-level predicates are located 
at the abstract level, while the individual locations within a city are placed 
at a lower level. Then an ordered refinement of an inter-city plan can be 
obtained by restricting the intra-city movements to be within the boundary 
of a city. 

Let i be an integer from 1 to n — 1. 

Definition 10.3.4 (Ordered Monotonic Property). For every i-th4evel 
solution n , if n has a refinement at level i — 1, then every refinement of II 
at level z — 1 is an ordered refinement of II . 

Obviously not every hierarchy satisfies the ordered monotonic property. 
Take the TOH domain for example. Consider a hierarchy where we assign 
the Onsmall and Onmed literals a higher criticality value than the Onlarge 
ones. And consider achieving a single goal literal 

goal = Onlarge(Peg3) 

Then no refinement of the abstract plan is an ordered refinement. To see this, 
observe that the abstract plan is in fact an empty plan. A refinement of the 
empty plan at level 0 is 




178 10. Hierarchical Planning 



^init 

I 

moveS(Pegl, Peg3) 

i 

moveM(Pegl, Peg2) 

i 

moves (Peg3, Peg2) 

1 

moveL(Pegl, Peg3) 

i 

^finish 

In this refinement, it is clear that in order to achieve the Onlarge(Peg3) goal, 
the abstract literals with predicates Onsmall and Onmed are all altered. 
Therefore, by the definition of ordered refinement, this hierarchy violates the 
ordered-monotonic property. Note that this conclusion holds for both total- 
order and partial-order refinements. 

The above example also reveals one of the shortcomings of ordered mono- 
tonic hierarchies. The requirement that no low-level refinements alter any 
high-level literal is in fact too strong; a seemingly harmless addition to 
a plan could make property invalid. On the other hand, despite its over- 
restrictiveness, it still doesn’t achieve one of its initial aim — to guarantee 
that there is no backtracking across abstraction levels. 

Take the simple travel example. Assume that we place all In-City literals 
at level 1, and all At-Loc literals at level 0. This hierarchy is monotonically 
ordered, since a refinement at level 0 would only patch up the intra-city 
movements without venturing out of the parimeter of a city. 

However, excessive backtracking between levels 0 and 1 can still occur. 
Any inter-city route that has been worked out at the abstract level could be 
found to be a dead end at level 0, due to unexpected road close-downs and 
traffic jams. In this case, re-planning for another inter-city route would then 
be necessary. In general, the ordered-monotonic property is neither necessary 
nor sufficient for ensuring that backtracking never occurs across abstraction 
levels. 

Downward Refinement Property. In response to this issue, Bacchus 
and Yang [13] identified a property as the operationalized version of the 
downward-solution property — the downward refinement property^ or the 
DRP. 

Definition 10.3.5 (Downward Refinement Property). Every abstract 
solution at level i can be monotonically refined to a solution at the next lower 
level, level i — 1. 

The implication of the DRP is straightforward: if a hierarchy satisfies the 
DRP, then during planning, we only need to keep one copy of an abstract 
plan. From the DRP we know that this plan will lead to a correct plan at 
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the next level. By induction, we also know that a concrete-level solution will 
eventually be found. Thus, the DRP could be used to dramatically prune a 
search space without loss of completeness. 

Just like the ordered monotonic property, and unlike the monotonic prop- 
erty, the DRP is not universal; it is not satisfied by every precondition- 
elimination hierarchy. Nor is it satisfied by every task-network hierarchy, 
which we will explain in Chapter 12. What we would like to do next is to 
study the behavior of an abstract planning system as its hierarchy approaches 
this property, in a probabilistic sense. We can then state the impact of a 
near-DRP hierarchy on planning efficiency, which should have a wider range 
of application than the DRP in its strictest form. 



10.4 An Analytical Model 

We now analyze the benefits of near-DRP hierarchies, by studying a proba- 
bilistic model of abstract search. The search space explored by a hierarchical 
problem-solver can be viewed as a tree generated by the abstraction hierarchy 
(see Figure 10.7). In this tree each node at level i represents a complete i-th 
level abstract solution. The children of a node represent all of the different 
refinements of that solution at the next (lower) level of abstraction. The leaf 
nodes are complete concrete- level solutions. The task in searching through 
the space of abstract solutions is to find a path from the root down to a leaf 
node representing a correct concrete-level solution. Each node on the path 
must be a correct i-th level abstract solution to the problem at hand and 
must be a refinement of the (i -f l)-th level abstract solution represented by 
its parent. That is, we are searching for an abstract solution that can be 
successfully refined through the levels of abstraction down to the concrete 
level. 

The work in searching this tree comes from the work required to find the 
solution at each node. This solution is a refinement of the solution that has 
already been found at the node’s parent. Hence, finding it simply involves 




Fig. 10.7. Our analytical model: a tree of abstract solutions, with a branching 
factor B and a depth n 4- 1 . 
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solving all of the subproblems that arise when the parent’s solution is moved 
down to this next lowest level. Since the number of subproblems increases as 
we move down the tree, the total amount of work will depend both on the 
number and depth of the nodes examined during search. 

More specifically, the root of the tree represents a special length-one solu- 
tion to every problem: a universal solution. Its presence is simply a technical 
convenience. The levels of the tree are numbered {n, . . - , 0} with the root 
being at level n and the leaves at level 0. Hence, discounting the universal 
solution at level n, our abstraction hierarchy has n levels. 

The tree of abstract plans will have a branching factor that, in general, 
varies from node to node. This branching factor is the number of (i — 1)- 
th level refinements possible for a given level-i solution, i.e., the number of 
children a node at level i has. Let the maximum of these branching factor 
be B. For simplicity we will use B as the branching factors for all nodes in 
the tree. Note that B has no straightforward relationship with the branching 
factor generated by the operators when searching for a specific solution, which 
we will denote by lower case h. 

Under the assumption that the subproblems are independent, the number 
of refinements of a given i-th level solution, B^ will be the product of the 
number of different solutions to each of the subproblems. Hence, it might be 
thought that B would grow exponentially as we move down the tree as the 
number of subproblems is growing exponentially. However, as we move down 
the levels of abstraction the subproblems become more and more constrained: 
at each level the solutions must preserve more and more conditions achieved 
at the higher levels of abstraction. Hence, the number of different solutions 
to the subproblems drops as we move down the tree. The exact balance 
between these two effects is difficult to determine, but we have found that our 
assumption of a constant branching factor B yields a model that is supported 
by empirical results. 

10.4.1 Assumptions 

At this stage for simplicity and ease of presentation we carry through our 
analysis under some assumptions. In our presentation, we don’t restrict our- 
selves to total-order plans and to total-order refinements. We assume that 
each i-th level plan, when taken to a lower level to refine, will contain certain 
new subproblems. A schematic diagram for total-order refinement is shown 
in Figure 10.8, and one for partial-order refinement is shown in Figure 10.9. 
In precondition-elimination hierarchies, these subproblems arise due to the 
newly introduced preconditions at level i — 1. For example, in the simple 
travel domain a new precondition might be a new location for the agent to 
arrive at, within a city. 

The amount of work required to solve each subproblem individually is 
assumed to be 0(5^). Here b is the effective branching factor of search. In 
forward-chaining, total-order planning b is simply the number of applicable 
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Refinement 

direction: 



^f 




Fig. 10.8. Subproblems in a total-order refinement 



refinement 







Fig. 10.9. Subproblems in a partial-order refinement 



operators per state. In backward-chaining, partial-order planning, b is derived 
from equation (3.5): 

~ ii^new + Bold) * 

where Bnew is the number of new steps added in each search node, Bold is 
the number of existing plan steps used for achieving each precondition. P is 
the number of preconditions for each plan step, t the number of threats per 
node expansion, and r the number of ways to resolve each threat. 

Our assumptions are as follows. 

Regularity. 

We assume that it takes approximately k new plan steps to solve every sub- 
problem, where k is constant across abstraction levels. This is referred to 
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as the regularity assumption. Refining a solution to the next level amounts 
to solving all subproblems, hence the refined solution will be k times larger. 
Since the root is a solution of length 1 , this means that the solutions at level i 
are of length and that the concrete-level solution is of length ^ which 
we also denote by i. 

As this assumption degenerates, the value of abstraction degenerates. If 
we end up having subproblems which require solutions of length 0(JL) instead 
of Oik) = then solving them will require search of 0(6^) where h 

is the branching factor generated by the operators. This is no better than 
search without abstraction. 

Independence Assumption. 

We assume that the individual subproblems can be solved without significant 
interaction. 

If, say, s subproblems interact we will have to search for a plan that solves 
all of them simultaneously. Such a plan would be of length 0{sk) and would 
require 0{b^^) search. As sk approaches i we once again degenerate to search 
complexity of 0{b^) and abstraction yields no benefits. 

In total-order planning the subproblems are naturally presented in the 
form of gap problems between every pair of abstract plan steps. In partial 
order planning, two subproblems do not have to follow each other in time se- 
quence; they can be left unordered in a partial-order. In this sense, the inde- 
pendence assumption places enough confidence in the quality of the abstrac- 
tion hierarchy that, if two abstract plan steps are unordered, then the details 
concerning these two steps — those arising from their new preconditions — are 
largely non-interfering, too. 

These two assumptions, regularity and independence, correspond to ig- 
noring that the abstract solution might fail to provide an effective decom- 
position of the problem. The brief discussion above indicates, however, that 
when these assumptions fail, hierarchical problem-solving can quickly degen- 
erate to being worse than non-hierarchical problem-solving. Hence, these two 
assumptions are required before we can obtain any interesting behavior from 
the abstraction hierarchy. 

Existence of a Solution. 

We assume that a concrete-level solution exists. This assumption is only 
needed to simplify our presentation. Actually, the anal 3 rbical model we de- 
velop will also cover the case where this assumption fails. 

Upward Solution Property. 

We assume that the upward- solution property holds of the hierarchy. This 
property simply says that if a concrete-level solution ilo exists then there 
exists a sequence of abstract solutions ending with ilo, {^m • • • ^ such 
that each II i is an i-th level abstract solution, and iJi_i is a refinement of 
Ili. That is, there is a sequence of refinements yielding a concrete-level plan. 
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Shortest Solution. 

We assume that the abstract planner can find the shortest concrete-level solu- 
tion for a problem. Let the solution length of the concrete-level optimal plan 
be 1. By the above assumptions, I = k'^. 

This assumption is also crucial in establishing the efficiency argument. 
If in the abstract space there are several alternative abstract plans, all with 
the same cost, then when we search in the abstract space we are blind as 
to which abstract plan to choose for refinement. In the worst case we could 
be led to an exponentially longer solution at the concrete level, leading to 
a degeneration of hierarchical problem-solving. This phenomenon was first 
pointed out by Backstrom and Jonsson [16]. 

Most of the above assumptions coincide with those considered by Knob- 
lock [80], although they are not identical. Knoblock’s analysis showed that 
with the above assumptions and when the DRP holds, using abstraction 
can yield an exponential-to- linear reduction in planning time. This result 
corresponds to the best-case behavior of an total-order abstract planning 
system. Below, we analyze the average-case search behavior for our general 
model. 

10.4.2 The Probability of Refinement 

If a hierarchy has the DRP then every solution at abstraction level i can be 
refined to a solution at abstraction level i—1. This means that once we have 
found a path down to level i, whose terminal node represents a correct i-th 
level solution, we are assured that this node has a child representing a valid 
1-th level solution. That is, we are assured that the node can be refined to 
the next lower level. Hence, we need never reconsider the initial part of the 
path, we just need to extend it until we reach a leaf: there is no backtracking 
across abstraction levels. 

A reasonable way to examine the behavior of hierarchies in which the 
DRP fails is to assign a probability, p, to the event that a given i-th level 
solution can be refined to level i—1. The DRP now corresponds to the case 
p = 1. When p < 1, however, we might build a path of correct solutions from 
the root down to a node at level i, and then find that this node is not refinable 
to the next level. This will force a backtrack to the penultimate node at level 
i-\-l to find an alternate level i solution, one which is refinable. This may 
cause further backtrack to level 2-1-2, or search may progress to lower levels 
before backtracking occurs again. 

Since we are assuming that a concrete- level solution exists, and that the 
upward solution property holds, we know that there is at least one path of 
correct solutions in the tree from the root to a leaf node. Hence, although the 
worst case will require an exhaustive search of the tree, search will eventually 
succeed in finding a good path. 

What we wish to accomplish, then, is to determine the average case com- 
plexity of search in abstraction hierarchies in which (1) the probability that 
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a given node in the abstraction search tree can be refined is p, and (2) there 
is at least one good path, i.e., a path of good nodes, from the root to a leaf 
in the tree. 

If a node is refinable to the next level, it will have B children, under 
our assumption that the number of refinements be treated as a constant. 
Each of these children is itself refinable with probability p. During search if 
we encounter a node that is refinable we would have to examine all of the 
subtrees headed up by its B children before we can conclude that it is a dead 
end. On the other hand, if the node is not refinable we can backtrack right 
away. Hence, as far as search is concerned, a node that is refinable has B 
children, while a node that is not refinable has no children. 

Analytically, however, we can find the average case complexity of search- 
ing such trees by considering randomly labeled complete trees. A complete 
abstraction tree is simply a tree in which every node, refinable or not, has B 
children and which has height n + 1. Hence, it has 



N = 



^n+l _ I 

B -I 



nodes. We randomly label each node in this tree as being refinable {good) 
with probability p, or not refinable {had) with probability 1 — p. Each of 
the 2^ distinct trees that can be generated by this process has probability 
p5(i _ p)^“^, where g is the number of good nodes in that tree. Now the 
average case complexity for searching this collection of trees is simply the 
sum of the search effort required for each tree times the probability of that 
tree. The average case complexity for searching all of these trees is the same 
as the average case complexity of searching the original set of trees in which 
a bad node has no children. Each original tree in which bad nodes have no 
children represents a set of complete trees: each one generated by a different 
labeling of the subtrees under the bad nodes. The probability of the original 
tree is exactly the same as the sum of the probabilities of the complete trees 
in this set. 

We wish to restrict our attention to the case where each tree contains at 
least one good path. That is, we assume that a solution exists at the concrete 
level. When we generate the set of differently labeled complete trees, some of 
them will not contain a good path from the root to a leaf. We remove these 
trees, and re-normalize the probabilities of the remaining trees so that they 
sum to 1. That is, we take only conditional probabilities. 



10.4.3 Analytical Result 

Let GoodTreeWork(i) be the expected amount of computation required to 
search a good subtree with root at level i, i.e., a subtree which contains at 
least one good path from its root to a leaf. To examine a good tree we have 
to expand its root node. Then we must search the subtrees under the root. 
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Table 10.8. Asymptotic search complexity for different regions when a solution 
exists 
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looking for a good subtree rooted at the next level. Once we find such a 
subtree we never need backtrack. 

The average-case time complexity of abstract planning is GoodTreeWork(n), 
which can be calculated by the following theorem. 

Theorem 10.4.1 



GoodTreeWork(n) = < 



0{h^k^-^) 

0{h^k^-^) 

0{h^k^-^n) 



p=l. 
p < 1/B 
p = 1/B, 
1/B < p < 1 



( 10 . 1 ) 



The proof requires familiarity with the mathematics of branching processes 
[9] . Interested readers may find a thorough proof in [13] . 

With the result from this theorem, we can now discuss average-case search 
complexity in different regions of refinement probability. There are two cases 
to consider: hierarchies with a constant number of levels and those with a 
variable number. In certain domains we can make n, the number of abstrac- 
tion levels, vary with i, the length of the concrete solution. For example, in 
the Towers of Hanoi domain we can place each disk at a separate level of 
abstraction [80]. In other domains, e.g., blocks world, it is not so easy to con- 
struct a variable number of abstraction levels, and n is generally fixed over 
different problem instances. 

Since each refinement multiplies the length of the abstract solution by k, 
we have that the concrete solution will be of size k'^, i.e., k'^ = i. We want 
to express our results in terms of i. If we can vary n with i then we can 
ensure that k remains constant and we have that n = In this case, 

b^ will become a constant. Otherwise, if n is constant, k = will grow 
slowly with £. In this case, b^ = b^ grows exponentially with i, albeit much 
more slowly than b^ (cf. [80]). This essential difference results in different 
asymptotic behavior for the two cases, n variable and n constant. Table 10.8 
gives the results of our analysis for these two cases expressed in terms of the 
length of solution £. 

We conclude from this table the following key points: 

1. Non-hierarchical search requires 0(6^); hence, it is evident from the table 
that when 0 < p < 1/B and when p = 1 abstraction has a significant 
benefit. Our result for p = 1 agrees with that of Knoblock [80]: here 
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we have the DRP and all of his assumptions hold. Our results for the 
region 0 < p < 1/B, extend his analysis, and indicate that abstraction is 
theoretically useful when the probability of refinement is very low. 

2. The region 1/5 < p < 1 corresponds to the worst region for search com- 
plexity. For variable n, in this region we increase by a factor, 0((p5)’^), 
that is exponential in n. In these regions it is not always advantageous 
to increase the number of abstraction levels n, especially in the region 
l/5<p<l. Asp increases in this region search first becomes harder 
and then easier. This can be seen from equation (10.1). 

Intuitively, what is occurring in this region is that the bad subtrees are 
becoming increasingly difficult to search. We have to search a larger and 
larger proportion of a bad subtree before we can recognize that it is bad. 
At the same time, as p grows, the number of bad subtrees we have to 
search is decreasing. These two trends fight each other, with the total 
work required first growing and then shrinking, until we reach p = 1 
where the number of bad subtrees we have to search falls to 0. 

Our analysis also tells us that if the number of possible refinements for 
an abstract solution, 5, is large, then searching the abstraction tree is more 
expensive in the worst region 1/5 < p < 1. This is to be expected: the ab- 
straction tree is bushier and in this region we have to search a significant 
proportion of it. Also of interest is that 5 does not play much of a role out- 
side of this region, except, of course, that it determines the size of the region. 
Hence, if we know that the DRP holds or if the probability of refinement 
is very low, we do not have to worry much about the shape of the abstrac- 
tion tree. However, without such assurances it is advantageous to choose 
abstraction hierarchies where abstract solutions generate fewer refinements. 
For example, this might determine the choice of one criticality ordering over 
an alternate one in precondition-elimination style abstraction. 

Another interesting result of our analysis is that search is most complex in 
the middle region. When the probability is low that an abstract solution can 
be refined, more abstract solutions need to be examined before a refinable 
one is found. However, not much work is required to detect unrefinability. 
On the other hand, when the probability of refinability is high, not too many 
abstract solutions need be checked before a good one is found. The worst 
case is in the middle. There a significant fraction of the abstract solutions 
are unrefinable, and it can take a great deal of work to discover that they are 
unrefinable. The existence of such a phase boundary agrees with empirical 
studies of Gheeseman et al. [30], who found that the hard cases of many 
problems tend to cluster in the phase boundary between very many solutions 
and very few solutions. 
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10.5 Open Problems 

Following the aforementioned results in abstraction, a few critiques emerged. 
With these, the study of abstraction, like any other scientific discipline, is 
hkely to undergo another cycle of evolution. 

Of the critiques, the first is by Lansky and Getoor [87] , who argued that 
many of the formal properties, such as the ordered-monotonic property, are 
in fact heavily dependent on a lack of subgoal-interactions between levels 
of hierarchies. This strict condition is often the reason to collapse an entire 
hierarchy into a single level, thereby diminishing the value of abstraction 
altogether. What is needed is a hierarchy that can tolerate a certain number 
of interactions between levels, while keeping the computational complexities 
well under control. 

Another critique is offered by Smith and Peot [123]. They demonstrated 
that the ordered-monotonic property is in fact not able to handle interactions 
arising from resource allocations. As we will see in the next chapter, part of 
their criticism can be dealt with under a property we call near-DRP. But the 
issue still largely remains; it is likely that some other unknown properties 
might exist that can characterize a different type of interactions, such as 
resource contention. 

A third criticism is made by Bergmann and Wilke [20]. They argued 
that in order for an abstraction planner to be truly efficient, it is crucial to 
provide an explicit refinement function for plans between successive levels in 
a hierarchy, and that at different levels, separate planning languages are used. 
In their work these refinement functions are provided by the user, and their 
empirical results have shown that having an explicit refinement function could 
make the planner dramatically more efficient than one with the precondition- 
elimination type of abstraction. 

Finally, Backstrom and Jonsson [16] have provided an analysis of hier- 
archical planning where the shortest-solution assumption is removed. They 
showed that in this case using abstraction it is possible to obtain an ex- 
ponentially more costly solution plan (see Section 11.5 for a more detailed 
discussion). 



10.6 Summary 

In this chapter we provided a framework in which to study abstraction. We 
described the precondition-elimination type of abstraction as a case study, 
and outlined a number of properties with which one could judge the quality 
of an abstraction hierarchy. Finally, we performed an average-case analysis 
of the value of abstraction, showing that, under a number of assumptions, 
the refinement probability of a hierarchical planner largely determines the 
efficiency of abstract problem solving. 




188 10. Hierarchical Planning 

10.7 Background 



Korf gave an early account of the value of abstraction in search [83]. Several 
books on hierarchical problem solving are also available. Knoblock’s book 
[80] presents a comprehensive coverage, both theoretical and empirical, of 
the Alpine system. Chapter 4 of the book by Allen, Kautz, Pelavin, and 
Tenenberg [6] discusses inheritance hierarchies and various properties associ- 
ated with precondition-elimination hierarchies. Wilkins’ book [147] discusses 
various aspects related to hierarchical task network hierarchies in the con- 
text of the SiPE planner. Sacerdoti’s book [114] makes good reading on the 
historical background in this topic. Holte et al. [67] investigated hierarchical 
A*, a forward-chaining, total-order planner using abstraction. Hierarchical 
problem solving is also well explored in theorem proving. See [58] for a good 
introduction. 

The material in this chapter came from several sources. Sections 10.1 
and 10.2 are adapted from a Computational Intelligence article with Tenen- 
berg and Woods [160]. Section 10.3 is adapted from articles appearing in 
AAAI90 (with Tenenberg) [159], AAAI91 (with Knoblock and Tenenberg) 
[79], AAAI92, and IJCAI91 (with Bacchus) [12, 11]. Section 10.4 is adapted 
from an Artificial Intelligence journal article with Bacchus [13]. 
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Having established a theoretical understanding of when hierarchical planning 
can improve planning efficiency, we now discuss how to ensure the downward 
refinement property^ or the DRP. Recall that the DRP holds for a hierarchy 
if every abstract solution can be refined to a concrete solution, given that 
a concrete solution exists. In this chapter, we concentrate on precondition- 
elimination hierarchies, for which we provide a useful syntactic condition that 
is sufficient to guarantee the DRP. Hence, in certain cases we can detect if 
a hierarchy is “good” in the sense that, under the assumption of easy and 
independent intermediate goals, we know, via the analytical models, that 
such hierarchies yield a significant speed-up over non- hierarchical planning. 

With the syntactic condition in place, we also design a hierarchy genera- 
tor called Hipoint. This system builds on the Alpine system of Knoblock 
[80], which is an automatic hierarchy generator for precondition-elimination 
hierarchies. HiPOiNT takes the hierarchy suggested by Alpine and improves 
it using information gathered during a testing phase. These improvements 
are based on the results of our analysis as well as the syntactic condition 
we develop. Our empirical results demonstrate that Hipoint can generate 
hierarchies that offer a significant improvement in performance over non- 
hierarchical planning. 



11.1 Syntactic Connectivity Conditions 

To know whether the DRP holds for a given precondition-elimination hier- 
archy, one way is to check, for each pair of operators that could possibly 
appear in sequence in a plan, whether it is possible to “connect” the lower 
level instantiations of the operators by a plan segment. We hope that this 
plan segment will leave all upper-level causal links intact, thus producing a 
monotonic refinement. 

Clearly, it is computationally intractable to check for the existence of 
plan segments of any arbitrary length. So, instead we realize our test for 
this condition by checking for solutions of length k. This means that we only 
need 0(6^) work to perform this test for each pair of operators. We keep k 
fixed and small so that the test can be run efficiently. An alternate way of 
realizing the test is to simply search for a solution under a fixed time bound. 
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Both of these approaches mean that the syntactic test is only a sufficient, not 
necessary test for the DRP. 

Suppose that we are given a precondition-elimination hierarchy with n 
levels. Assume that level 0 corresponds to the concrete level. The k-ary con- 
nectivity test is run on a pair of operators a\ and a 2 and is also dependent 
on the level of abstraction i. The test is in the form of an implication. Hence, 
a pair of operators will pass the test if they fail to satisfy the first condition, 
i.e., the antecedent. To describe this condition, we first introduce a notion for 
applying an operator to a set of literals. Let a be an operator, and L a set 
of literals. ol{L) is a set of literals obtained from L, by removing all literals 
in L which are not consistent with an effect of a. That is, 

a{L) = L - {/ I / G L and -Z € Eff(a)} 

For the antecedent test the conditions are 

Definition 11.1.1 (fc-ary Antecedent Test). Two operators ai and a 2 
satisfy the k-ary antecedent condition if 

1. Abs(i, ai(Pre(ai))) U Abs(i, Pre(a 2 )) does not contain both a literal and 
its negation; and 

2. Abs(i— 1, Pre(a 2 )) ^ Abs(i, Pre(a 2 )). 

By the first condition ai and 02 are operators that could appear in sequence 
in an i-th level plan, and by the second, 02 will create a subproblem, so that 
there is the possibility of a problem when refining to this level. 

If ai and 02 fail this condition, the fc-ary connectivity test returns success; 
this pair of operators will not cause a problem during refinement. If the 
antecedent test is satisfied, the operator pair must then satisfy the consequent 
test, which is as follows. 

Definition 11.1.2 (A;-ary Connectivity Test). Given a pair of operators 
ai and 02 that satisfy the k-ary antecedent test, there must exist a sequence 
of k, i— 1-level abstract operators Pi,. . . such that: 

1. None of the P^ adds or deletes literals with criticality higher than i—1, 
and 

2. The sequence of operators p^ is a solution to the problem 

(Abs(i-1, ai(Pre(oi))), Abs(i-1, Pre(a 2 ))). 

If the consequent test succeeds, then we return success, otherwise failure. 
We can generalize the test to all operators in the domain: 

Definition 11.1.3 {k-ary Necessary Connectivity). Let Oyi be the set 

of operators whose effect lists contain at least one literal with criticality value 
greater than or equal to i. If every pair of operators a\, 02 from 0>i passes 
the k-ary connectivity test, then levels i and — are k-ary connected. If every 
pair of levels i, — in the hierarchy is k-ary connected then we say that the 
hierarchy satisfies the condition of k-ary necessary connectivity. 
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Fig. 11.1. A plan segment and its refinement 

Intuitively, A:-ary necessary connectivity is saying the following. Since any 
partial-order plan can be considered as sequences of total-order plans, we can 
concentrate on the most elementary component of a total-order plan — a pair 
of operators. For every pair of operators, if they might be sequenced in a plan 
at level i (the antecedent test), there must exist a sequence of i—1 abstract 
operators j3^ that can correctly solve any subproblem that might result from 
the refinement of a\ and a 2 to the next lower level, i—1 (the consequent 
test; see Figure 11.1). Any subproblem at abstraction level i—1 would have 
to start from a state that must include all the weakest conditions established 
as a result of applying a\. This state is specified by the application of a\ 
to its own preconditions, i.e., ai(Pre(ai)), and we simply have to take its 
i—1 abstraction. Similarly, the subproblem would have to end in a state that 
achieves the i—1 preconditions of operators a 2 - This argument can easily be 
formalized to yield: 

Theorem 11.1.1 k-ary necessary connectivity is sufficient to guarantee the 
DRP. 

As we increase the parameter k we increase the complexity of performing 
this test. The complexity increases exponentially with k as we are searching 
through the space of operator sequences of length k. 

Although we can keep each individual run of the fc-ary connectivity test 
at a reasonable level of complexity, by keeping k small, we still have to test 




192 11. Generating Abstraction Hierarchies 



every pair of instantiated operators to determine necessary connectivity. This 
might be quite a large number, although various techniques can be applied 
to minimize the number of tests we need. For example, the operator variables 
are subject to type constraints so not all instantiations are legal; similarly, if 
two constants have exactly the same properties we do not have to use both 
in our instantiations. However, we will not delve into such details here. Our 
main use of necessary connectivity will be to estimate refinement probabilities 
by subjecting random pairs of operators to the connectivity test. 

In practice, however, it may be too strong to require that every abstract 
plan have a monotonic refinement. In that case, we can relax this requirement 
and consider instead hierarchies that are close to having the DRP. In par- 
ticular, we can say that a hierarchy is near-DRP if it satisfies the following 
conditions: 

1. Let the refinement probability for level i be pi, z = 0, 1, . . . , n. Let 0 be a 
user-defined threshold, where 0.5 < 0 < 1. We require that pi > 6. That 
is, the refinement probabilities must be no less than a given threshold. 

2- Pj ^ Pi for all j > i. That is, the refinement probabilities must 
be monotonically increasing as we move down the levels of abstrac- 
tion. 

Both of these conditions are motivated by our analysis in the last chapter. 
As discussed in Chapter 10 sufficiently high refinement probabilities will re- 
sult in less search, and typically increasing refinement probabilities are more 
effective. 



11.2 Hipoint 

A good hierarchy should have the ability to avoid interactions with higher- 
level achievements, and it should ensure that for every abstract plan a low- 
level refinement exists with high probability. In this section we present an 
algorithm, HiPOiNT, that automatically constructs abstraction hierarchies 
possessing both of these properties, namely, the ordered monotonic property 
(OM) and the near-DRP. We would like to build on hierarchies with the OM 
property, because, due to its restriction that no low-level refinements can 
change a high-level literal, a hierarchical planner need not check on any of 
its abstract-level causal links to see if they are violated. The OM hierarchy 
guarantees that they will not. Thus, considerable computational gain can be 
achieved by using an OM hierarchy. 

The algorithm Hipoint is presented in Table 11.1. Informally, Hipoint 
takes as input a set of operators, initial states, and goal states of a domain. 
First it invokes Alpine [80] to generate a partially-ordered graph that rep- 
resents a set of hierarchies with the ordered monotonic property (OM) for 




11.2 Hipoint 193 



Table 11.1. The Hipoint algorithm for creating a hierarchy 



Input: A set of operators O, initial states init and goal states goal 
Output: A criticahty assignment to the predicates, such that the abstraction 
hierarchy satisfies the OM property and is close to being near-DRP. 

Algorithm Hipoint( 0, init, goal) 

1. graph := Alpine{ 0, init, goat); 

2. for every pair of nodes m and nj in graph, such that 

3. there is no path from Uj to Ui in graph, do 

4. prob{ni,nj) := FlND-PROBABlLlTY(ni, rij, O) 

5. end for 

6. graph := COLLAPSE-NoDES{graph,prob); 

7. hierarchy := AuGMENTED-ToP-SORT(^rap/i,pro6); 

8. ret urn (hierarchy); 



the given domain. Recall that the property states that with the abstraction 
hierarchy, every refinement leaves all literals in the higher levels intact. 

Each node in the graph represents a set of predicates that should be 
assigned the same criticality in an abstraction hierarchy. An arc from a node 
rii to rij denotes that plans for achieving subgoals whose predicates are in 
the set rij will not affect any predicates in the set n^. That is, if we have an 
abstract plan involving the achievement of literals whose predicates are in rii, 
then we can refine that plan to achieve literals in rij without affecting any 
of the rii literals. The graph has the property that every total order of the 
nodes that extends the partial order, represents a hierarchy with the Ordered 
Monotonic Property. 

Next, steps 2-5 of the algorithm assign an estimated refinement probabil- 
ity to every pair of nodes rii and rij such that crit{rii) > crit{rij) is allowed 
by the partial order defined by the graph. Step 6 processes the nodes in the 
graph using the additional information provided by the estimated refinement 
probabilities. The procedure Augmented-Top-Sort returns a criticality 
assignment to each predicate, such that the resultant hierarchy has the OM 
property and is close to being near-DRP. 

We cannot guarantee near-DRP for two reasons. First, we are only able 
to obtain estimates of the refinement probabilities, so the true refinement 
probabilities might fail to satisfy the near-DRP property. Second, the OM 
property forces the placement of some hterals above others. Since it does 
not consider refinement probabilities, the placement of literals forced by OM 
might result in the violation of near-DRP. HiPOiNT attempts to find the 
best hierarchy, with respect to being near-DRP, among those hierarchies that 
satisfy the OM property. The hierarchy is then returned as the output of the 
algorithm. 
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Below, we explain in detail each of the major components of Hipoint, 
with the help of an extension of the simple robot domain. In this domain 
there is a robot and a number of connected rooms between which the robot 
can move. Between any two rooms there may be a door, which can be 
open or closed. In addition, there are also a number of boxes, which the 
robot can either pull or carry from one location to another. Figure 11.2 
shows one configuration of the domain. The operators in this domain are 
presented in Table 11.2. There is one additional operator not shown in 
the table, caxry-thru-door(?b, ?d, ?r2, ?r2); this operator is identical 
to pull-thru-door except that the box, ?b, must be Loaded instead of 
Attached. 

Our representation for this domain includes the following predicates: 
Box-Inroom (?b,?r) representing that box ?b is in room ?r; Attached(?b) 
representing that box ?b is attached to the robot; Loaded (?b) representing 
that box ?b is loaded onto the robot, Open(?d) representing that door ?d is 
open. In addition, there are also a number of type predicates: Isdoor(Door) , 
Openable(Door), Connects(jDoori 2 , i^ 2 )- 

11.2.1 Alpine 

Alpine is an abstraction-hierarchy generation algorithm designed and imple- 
mented by Knoblock [80]. Given the operator definitions for a given domain, 
Alpine constructs a partially ordered graph of the literals. Each node in the 
graph denotes a set of literals that are to be assigned the same criticality 
value in the final hierarchy. If a node ni precedes Uj in the graph, i.e., if 
there is a path from ni to Uj in the graph, then we cannot place rij above 
Ui in the final hierarchy; the criticality value of ni must be greater than or 
equal to the criticality value of rij. The algorithm ensures that every total 
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Table 11.2. Operators for the robot-box domain 



Preconditions 


Effects 


pull-thru-door (?b,?d,?rl.?r2) 
Isdoor(?d), Isbox(?b), 


Box-Inroom(?b,?r2), 

-iBox-Inroom(?b,?rl) 


attach-box(?b) 

Isroom(?rl), Isroom(?r2), 
Connects(?d,?rl,?r2), Attached(?b), 
Box-Inroom(?b,?rl), Open(?d) 
Isbox(?b), -iAttached(?b) 






Attached (?b) 


load-box(?b) 

Isbox(?b), -iLoaded(?b) 
open-door (?d) 


Loaded(?b) 


Isdoor(?d), Openable(?d), 
-1 Open(?d) 


Open(?d) 



order supported by the graph will yield a hierarchy that has the ordered- 
monotonic property, whereby every refinement of an abstract plan leaves all 
the higher- level literals unchanged. 

The core of the Alpine algorithm is a syntactic restriction formalized in 
[79]: 

Definition 11.2.1 (Ordered Restriction). Let O he the set of operators 
in a domain. Let Pa be the preconditions of a that can he either added or 
deleted by some operator. Then a criticality assignment satisfies the ordered 
restriction if'ia £0,^p E Pa, and'iei^e 2 € Eff(o;), 

(1) crit{e\) = crit{e 2 ), and 

(2) crit{ei) > crit{p). 

That is, all the effects of an operator are required to have the same criticality, 
and that criticality must be at least as great as the operator’s changeable 
preconditions. It was shown in [79] that if the assignment of criticality values 
satisfies this restriction then the hierarchy will satisfy the OM property. 

The Alpine algorithm implements this restriction, generating a graph 
representing partially ordered collections of literals. The Alpine system then 
uses this graph to compute a particular total order and resulting hierarchy, 
using a collection of heuristics to pick the total ordering. The algorithm de- 
pends only on the operator definitions, not on the goals and initial situations. 
However, Knoblock has shown how to modify this algorithm so that it can 
generate problem-specific hierarchies that take into account a particular goal 
and initial state. This often results in a finer grained hierarchy [80]. 

We illustrate the algorithm via the robot-box example. When applied to 
the operators in this domain, Alpine generates the graph shown in Fig- 
ure 11.3. It is clear that there are six possible total orders that could result 
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Table 11.3. Robot-box domain hierarchy generated by Alpine 



Criticality 


Predicates 


4 


Isdoor, Openable, Connects, Isbox, Isroom 


3 


Box-Inroom 


2 


Attached 


1 


Loaded 


0 


Open 




Fig. 11.3. Robot-box domain graph generated by Alpine 



from this graph, corresponding to the predicate Box-Inroom followed by per- 
mutations of open, Attached, Loaded.^ Each total order corresponds to a 
different hierarchy. For example, for the total order, 

Box-Inroom --< Attached Loaded ^ Open 
the corresponding hierarchy is shown in Table 11.3. 



11.2.2 Probability Estimates 

If there is a path from to rij in the Alpine graph, we cannot place the 
literals in rij at a higher level of abstraction than the literals in rii^ without 
violating the OM property. However, for all pairs rii and rij that do not have 
this constraint we have a choice; we can, if we wish, place above rij in the 
final hierarchy. The next step of the HiPOiNT algorithm is to determine the 
merit of such a placement. It does this by estimating the refinement proba- 
bility that would exist between the levels rii and rij if rii was in fact placed 
above rij. If node ni was placed before n^, then all of the predicates in rii 
would be placed at a higher level of abstraction than the predicates in rij. 

For all pairs of nodes rii and rij such that it is possible to place rii above 
rij, Hipoint calls FiND-PROBABlLlTY(n^, n^) to estimate the probability of 
refinement going from predicates in rii to predicates in rij. 



^ The type predicates, like Openable, are not shown on this graph. Since these 
predicates are not affected by any of the operators, they are always placed at 
the highest level by Alpine. 
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In its operation Find-Probability first locates sets of operators Oi and 
Oj that achieve literals in rii and rij respectively. It generates its estimate of 
the refinement probability from rii to rij by determining how often pairs of 
operators in Oi pass the connectivity test, as in Definition 11.1.2, where we 
consider the predicates in rii to be at level i and the predicates in rij to be 
at level i — 1. 

Consider a plan at level rii. Such a plan will typically have to achieve 
literals in n^, hence it will contain operators from Oi . To refine this plan to 
the level of rij, we will have to solve the subproblems generated by pairs of 
operators from Oi using operators from Oj . According to Definition 11.1.2 
this means testing how often the planning problem 

(Abs(i - l,o:i(Pre(ai))), Abs(i - 1, Pre(a 2 ))) 

can be solved using operators that affect only literals at level i — 1 and below, 
where a\ and a 2 are members of Oi . Since we are ignoring all other nodes, 
we ignore all preconditions of a\ and 0:2 that are outside of rii and n^, except 
for type preconditions which are always at the top of the hierarchy. 

For operators in O^, Generate- Random-Probs chooses at random a 
pair of instantiated operators that passes the antecedent condition of the 
connectivity test. It then computes the problem shown above, which is spec- 
ified by the consequent condition of the connectivity test. A whole collection 
of such pairs of operators are generated, and their resulting problems are 
returned as the set “random-probs” . 

Find-Probability then calls a planner Planner, which tries to solve 
these problems. The Planner algorithm could be implemented as either a 
forward-chaining, total-order planner, or a backward-chaining, partial-order 
planner. In line 5 of the Find-Probability algorithm, the Planner pa- 
rameter OjjOi specifies the operators that can be used to solve the problem. 
This is the set of operators Oj with those operators that affect literals con- 
tained in rii removed, i.e., all operators also in Oi have been removed using 
a set-difference operation. Each problem is solved under a fixed solution- 
length bound, this implements the “A:- ary” limit part of the connectivity test. 
Find-Probability accumulates a count of the number of times a solution 
is found. 

The frequency of success serves as an estimate of the refinement proba- 
bility, as we know that if two operators pass the test then the gap subproblem 
they generate during refinement can be solved. If Generate- Random-Probs 
fails to find any random problems, then it has failed to find any operators 
with level rij preconditions among the set of operators that can be sequenced. 
Hence, it is unlikely that any difficulties will be encountered during refine- 
ment, and we estimate the refinement probabilities as being 1. 

We now illustrate the procedure using our robot domain. Let rii be 
{Box-Inroom}, and rij be {Open}. The operator sets corresponding to the 
nodes are 
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Table 11.4. Algorithm Find-Probability 



Input: Operators O, predicate sets n*, rij. 

Output: A refinement probability value. 

Procedure FlND-PROBABlLlTY(ni, n^, O) 

1. Oi := Find- Operator s{0 ,rii)\ 

2. Oj := Find- Operators{0, Uj ) ; 

3. random-probs := GENERATE- RANDOM-PROBS(Oj, rij, rij); 

4. for every random problem {initial ^ goal) in random-probs do 

5. if PLANNER(mitia^, goal, solution-length-bound) = Success then 

6. success-count := success-count + 1 

7. end if 

8. end for 

9. if I random-probs I = 0 then 

10. prob := 1 

11. else prob ;= success-count/ 1 random-probs | 

12. end if 

13. return(prob); 



Oi — {pull-thru-door, carry-thru-door} 

and Oj = {open-door}. Both of the operators in Oi can be sequenced in 
any manner as long as the room the first operator takes us into is identical 
to the room the second operator moves us out of; this constraint is enforced 
at the ni level via the Box-Inroom precondition of these operators. Addi- 
tionally, we ignore the precondition of Attached (or, Loaded in the case of 
carry- thru-do or) as this precondition is in node other than Ui or uj and it 
is not a type predicate. 

So of those operators that can be sequenced, i.e., that pass the antecedent 
test, we will generate random problems that are instantiations of the following 
template arising from the connectivity test: 



Isdoor(?dl), Isbox(?6) 
Isroom(?rl), Isroom(?r2) 
Connects(?dl, ?rl, ?r2) 
Box-Inroom (?6, ?r2) 
Open(?dl) 



Isdoor(?d2) , Isbox(?6) 
Isroom(?r2), Isroom(?r3) 
Connects(?d2, ?r2, ?r3) 
Box-Inroom(?6, ?r2) 
Open(?d2) 



That is, the problem is to achieve the preconditions of the second operator 
in the state that results from applying the first operator to its precondition 
set, as specified by the connectivity test, for various instantiations of the op- 
erators. It is easy to see that this problem reduces to achieving open(?d2) 
for different instantiations of ?d2. A door can be opened with the opera- 
tor open-door(?d) without affecting any higher level literals, as long as it is 
Openable. So we see that the number of solvable random problems will cor- 
respond approximately to the proportion of doors that are Openable in the 
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Table 11.5. A matrix of refinement probabilities 



Predicates 


Box-Inroom 


Open 


Attached 


Loaded 


Box-Inroom 


0 


0.5 


1 


1 


Open 


0 


0 


1 


1 


Attached 


0 


1 


0 


1 


Loaded 


0 


1 


1 


0 



domain. Hence, the refinement probability returned by Find-Probability 
will depend on the probability of a door being openable. This agrees with 
intuition. If we place Open at a lower level of abstraction, the abstract level 
will be free to develop a plan ignoring the status of the connecting doors. If 
most doors can be opened this will generally not be a problem, but if most 
doors cannot be opened, most of the routes chosen at the abstract level will 
fail. That is, most of the abstract plans will not be refineable. 

The result of running Find-Probability on all eligible pairs of nodes is 
a matrix of estimated refinement probabilities. In one of our tests in which 
half of the doors were openable we obtained the matrix of values shown in 
Table 11.5. 

11.2.3 Collapsing Nodes with Low Refinement Probabilities 

Using the refinement probabilities the procedure COLLAPSE-NoDES processes 
the ALPiNE-graph. In particular, based on the threshold 9 used in the near- 
DRP condition. Section 11.1, it decides if two nodes should be collapsed into 
one. 

If the refinement probability for both orderings of these nodes is below 
then no hierarchy in which these nodes are on separate levels will be close 
to being near-DRP. That is, if prob(ni,nj) < 9 and prob(n^-,ni) < 0, as 
found by the Find-Probability routine, then we collapse these two nodes 
into one nij = (n^Urij). This means that when we assign criticalities using a 
total order produced from this graph the literals in ni and rij will be given 
identical criticalities. Therefore, to ensure we satisfy the constraints imposed 
by the original partial order we must also collapse all nodes that lie on any 
path between rii and rij into the new node riij. To collapse the nodes, we 
modify the graph by substituting all collapsed nodes by the new node riij 
and then all in-edges of the collapsed nodes become in-edges of the new node 
and similarly for the out-edges. The refinement probabilities to and from 
all the remaining nodes must be recomputed for the new node. To avoid 
doing this computation, we choose instead to use the average of the original 
probabilities as estimates for the new ones, so that for every other node n, 
we let 



prob(n,nij) = Average{prob(n, n^) | rii has been collapsed into 
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Table 11 . 6 . Robot box domain hierarchy generated by HiPOiNT 



Criticality 


Predicates 


4 


Isdoor, Openable 


3 


Box-Inroom 


2 


Open 


1 


Attached 


0 


Loaded 



and similarly for prob(n^j,n) for nodes n connected via out-edges. The col- 
lapsing process continues until no more nodes can be further collapsed. 

In robot and box domain the two nodes containing Box-Inroom and open 
will be collapsed to one level whenever the threshold 6 is greater than the 
proportion of openable doors. In this case the refinement probability from 
Box-Inroom to open falls below 9, and the opposite ordering of these nodes 
is impossible. 

11.2.4 Augmented Topological Sort of Abstraction Graph 

After collapsing nodes with low refinement probabilities, the procedure 
Augmented-Top-Sort computes a total order of the nodes in the resulting 
graph, using both the partial order relation and refinement probabilities as 
guides to compute the order. The procedure is a simple modification of a 
standard topological sort algorithm (see Chapter 4 for a description of the 
topological sort algorithm). Our augmented version of the algorithm simply 
orders the collection of nodes that are placed on the queue during any one 
step. That is, at each stage we add those nodes with no in-edges to the queue, 
but we add them to the queue so that the sequence of Find-Probability 
estimates between these nodes is ascending. For example, if at some state we 
are to add the nodes ni, ri 2 , and ns to the queue we would choose, e.g., the 
ordering 713, ni, ri2 if prob(ri3,ni) < prob(ni,n2). 

To illustrate the procedure, consider again the simple robot example. 
The graph generated by Alpine now has associated refinement probabilities, 
shown in Table 11.5. The augmented topological sort algorithm will place 
Open right below the Box-Inroom level, resulting in the hierarchy shown in 
Table 11.6. 



11.3 Empirical Results in the Box Domain 

In the last section, we have described the HiPOiNT algorithm for generat- 
ing abstraction hierarchies by augmenting the Alpine algorithm, taking into 
account the refinement probabilities between abstraction levels. To compare 
their performance, we have conducted a set of experiments in the box do- 
main, where we placed a time bound of 30 CPU minutes. For the refinement 




11.3 Empirical Results in the Box Domain 201 



probability estimate computation, a backward-chaining, partial-order plan- 
ner, AbTweak, is used. AbTweak is called with a solution length limit of 
five steps. The features of the domain are that we are able to change the 
refinement probability at each level individually, by changing the mixture 
of objects with different properties. For example, we can change the refine- 
ment probability between Box-Inroom and open by changing the proportion 
of openable doors. All systems and domains were implemented in Allegro 
Common LISP on a Sun4/Sparc Station. 

In our realization of the box domain we place two doors between every 
pair of adjacent rooms. Each door may or may not be openable. The robot’s 
task is to move boxes between the room by either carrying or pulling them. 

For domain instances where every door is openable, both Alpine and Hl- 
POINT generate the same abstraction hierarchy, shown in Table 11.3. There- 
fore the costs of actual problem-solving using the Alpine and Hipoint hi- 
erarchies are the same. The only difference is that HiPOiNT requires extra 
time to determine the refinement probabilities. For the domain where every 
door is openable, Hipoint takes 27 CPU seconds to generate the hierarchy, 
while Alpine took only 0.05 CPU seconds. However, to solve each problem, 
both systems take 4-5 seconds on average. Thus, as the number of problems 
grows, Hipoint’s initial cost is amortized away. 

We then changed the domain so that not all the doors were openable. In 
this case not every plan that ignores the status of the doors will be refineable, 
and Hipoint will place open higher up in the hierarchy, so that this condition 
can be tested earlier on, before more resources are allocated to solving the 
Loaded and Attached preconditions. In our test we set more than 50% of the 
doors to be openable, so HiPOiNT did not collapse any levels. The hierarchy 
it generated is shown in Table 11.6. Hipoint takes 75 seconds to generate 
this hierarchy, after checking 20 randomly generated problems. In contrast, 
Alpine is not able to alter its hierarchy in response to a change in the number 
of openable doors, so it generates the same hierarchy as before. 

To compare the qualities of the hierarchies generated by Hipoint and 
Alpine, we first ran AbTweak on six problems of varying sizes. AbTweak 
solves each problem by first using the hierarchy generated by Hipoint, and 
then by using the one by Alpine. The CPU time costs for both hierarchies, 
which do not include the time for generating the hierarchies, are shown in 
Table 11.7. The table demonstrates that as the planning problems get more 
complex, Hipoint is increasingly more efficient than Alpine. When the ini- 
tial cost for generating the hierarchies is taken into account, Hipoint might 
require a number of problems of small size before it can recover the cost of 
its more expensive hierarchy generation algorithm. However, for problems of 
large size, Hipoint outperforms Alpine by such a large margin that it can 
recover its initial cost of hierarchy generation in a single problem, yielding 
an immediate improvement in net problem solving costs. 
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Table 11.7. CPU time comparison between Hipoint and Alpine 
in the box domain 



Solution Length 


HlPOlNT(seconds) 


Alpine (seconds) 


0 


0.08 


0.08 


3 


0.55 


0.48 


6 


4.15 


6.55 


9 


11.17 


17.23 


12 


97.02 


419.02 


15 


229.30 


903.25 



As a further test we ran 42 test problems of equal length using the hierar- 
chies generated by HiPOiNT and Alpine, respectively. Figure 11.4 compares 
the accumulated CPU time of both systems over the same 42 problems. The 
time required by the algorithms to generate their abstraction hierarchies is 
also included in the values plotted. It is clear from the figure that the Hipoint 
hierarchy is soon able to recover from its initial cost, and after 2 problems 
of this size the net cost was lower than problem solving with the Alpine 
hierarchy. In this domain the Hipoint hierarchy is able to solve problems of 
this size almost twice as fast as the Alpine hierarchy, and as the previous 
table has shown, as the problems become longer so does Hipoint’s factor of 
improvement. 



CPU 

Seconds 




Fig. 11.4. Robot box domain tests 
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In several other domains we have also shown that HiPOINT is able to 
offer a significant improvement over Alpine, which serves to demonstrate 
the validity of our approach, and the importance of the DRP property. These 
other results are reported in [13]. To sum up, HiPOiNT has the following 
advantages over Alpine: 

1. As demonstrated by tests in the robot-box domain, when Alpine gener- 
ates a partially ordered graph of predicate sets, for hierarchy generation, 
its selection of a total order does not depend on refinement probabili- 
ties. Thus, it is possible that it may generate a hierarchy in which the 
refinement probabilities are poorly configured, e.g., where they are not 
increasing as we move down the hierarchy. In contrast, HiPOiNT is able 
to select a more intelligent total order by examining these probabilities. 

2. When the refinement probability is low, HiPOINT recognizes the need to 
collapse two or more levels. 

3. Although Hipoint requires more time to generate its hierarchy, it is clear 
that over a number of problem solving instances, and for problems with 
lengthy solutions, this cost is quickly paid off. 



11.4 Related Work on Abstraction Generation 

In the past, many abstract planning systems have relied on the user to provide 
an abstraction hierarchy [147, 132, 159]. One of the first systems that semi- 
automatically generated its own abstraction hierarchies was Abstrips[113]. 

To build an abstraction hierarchy, in addition to the domain specifica- 
tion, Abstrips also requires a user-defined partial order on the literals in 
the domain. It then tries to assign criticality values to the literals that are 
consistent with the given partial order. For each literal Z, Abstrips searches 
for a short plan that achieves I from a state where all literals before I in the 
partial order hold. If such a plan is found, then I is considered a detail, and is 
assigned a low criticality value. Otherwise, I will be assigned a high criticality 
value. This algorithm can be considered as a method for judging the quality 
of an abstraction hierarchy: a hierarchy is “good” according to Abstrips, if 
for every low-level literal Z, there is a short plan to achieve it from a state 
where all high-level literals are true. Pablo [31], a successor of Abstrips, 
can also be viewed in this way. 

At a first glance, our syntactic necessary connectivity condition, described 
in Definition 11.1.2, is similar to Abstrips since they both depend on finding 
short plans to achieve low level literals. However, there are some significant 
differences. 

First, our condition specifies that all low-level precondition literals of an 
operator must be simultaneously achievable, whereas ABSTRIPS only requires 
that each literal be achievable individually. However, in domains where inter- 
actions often occur, the existence of an individual plan for each literal does 
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not ensure the existence of a plan for the simultaneous achievement of all of 
the literals. 

Second, our necessary connectivity condition specifies that the low-level 
plan for achieving the preconditions of an operator should not violate any 
abstract conditions achieved at higher levels. This is in accordance with our 
notion of monotonic refinement. In contrast, A B STRIPS does not specify any 
restriction on the plan that achieves the low level literals. This means that 
when Abstrips searches for a refinement of an abstract plan it does not 
restrict itself to searching for monotonic refinements. This can significantly 
increase the length of the refinement, and can increase the cost of finding 
it. Also, during refinement the plan to achieve a low level literal might undo 
work accomplished at the higher level. 

In general, although Abstrips attempts to order the literals in such a way 
as to make its abstract plans easily refineable, it does not take into account 
all of the factors that affect refineability. Its techniques are mainly heuristic, 
and were developed without a formal analysis of the problem. 

It can also be noted that Abstrips only partially automates the ab- 
straction process; the quality of its hierarchies depends heavily on the user- 
supplied partial order of the literals. 



11.5 Open Problems 

Hipoint can easily be improved, but since this was not the focus of our work 
we were content with a simple implementation. 



The Shortest Solution Assumption 

When deriving the exponential speed-up of the DRP in the last chapter, we 
have assumed that a hierarchical planner using a depth-first search always 
descends on the shortest solution to the original problem. However, as pointed 
out by Backstrom and Jonsson [16], this might not always be the case. 

To see this, consider an extended TOH domain where we add an additional 
operator move-medium-aiid-large(?x,?y), for simultaneously moving both 
the medium disk and the large disk from a peg ?x to ?y. This operator is 
shown in Table 11.8. With this extension, and with the hierarchy ILMS, 

Table 11.8. An additional operator in the extended TOH domain 
Preconditions Effects Cost 

mo ve -medium-and-lar ge ( ?x , ?y ) 

Onlarge(?x), Onmed(?x) Onlarge(?y), Onmed(?y) 2.0 

-iOnsmall(?x), -iOnsmall(?y) ->Onlarge(?x), -iOnmed(?x) 

Ispeg(?x), Ispeg(?y) 
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there are now two interesting abstract plans at Onlarge level. The first plan 
is the usual inoveL(Pegl, Peg3). This abstract plan refines to a 7-step solution 
plan as obtained in the original TOH domain. 

The second abstract solution, move-medium- cLnd~laxge(Pegl, Peg3), has 
a cost of 2. Although the cost of this abstract plan is higher than the first 
one, the plan refines to a less costly concrete-level plan: 

init 

i 

moveS(Pegl, Peg2) 

1 

move-medium-and-large(Pegl, Peg3) 

i 

moveS(Peg2, Peg3) 

1 

goal 

This plan has a cost of 4. 

This example reveals one of the shortcomings of HiPOiNT and Alpine. 
When multiple abstract-level plans exist, a hierarchical planner using an ab- 
straction hierarchy generated by either algorithm would still have to pick one 
randomly in order to refine the plan to a concrete-level solution. It may be 
much too late before the planner realizes that it is working on a sub-optimal 
plan with very high cost. To solve this problem, one could define additional 
properties to strengthen the constraints placed on a near-DRP hierarchy. 
Or one could allow a certain amount of look-ahead during refinement of an 
abstract plan. 

Probability Estimate 

One way for Hipoint to be improved is for it to use our analytic forms in 
Chapter 10 directly to estimate the amount of work required to solve a prob- 
lem on candidate hierarchies. This would involve a more thorough search 
through the space of candidate hierarchies; for the searched candidates the 
analytical forms predicting their behavior could be evaluated using the in- 
formation gathered by Find-Probability. This would give a more accurate 
evaluation of the hierarchy’s worth than the approximations we used. 

Another improvement would be to extend Hipoint to handle individual 
planning problems rather than to try to construct a single hierarchy for all 
problems in the domain. Knoblock has demonstrated that problem-specific 
hierarchies can often display superior performance [80]. 

Threshold Value 

Another source of improvement is to obtain a better understanding of the 
threshold value 0. In this work, the value is set to 0.5 for near-DRP hierar- 
chies. However, there might be other better values depending on the domain 
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of planning. The appropriate value for 9 is likely to depend on the domain 
features. Finding the right value for 6 is therefore an excellent target for 
machine learning systems. 



11.6 Summary 

In this chapter we discussed a syntactic condition for testing if a precondition- 
elimination abstraction hierarchy possesses the DRP. The condition can also 
be used to estimate refinement probabilities. An algorithm has been provided 
that automatically constructs an abstraction hierarchy for a given domain. 
The hierarchies constructed have both the ordered monotonicity property 
and the near-DRP. Empirical tests have been presented that support our 
analytical model, and confirm the utility of our algorithm. 



11.7 Background 

Knoblock’s book presents a detailed description of the Alpine algorithm 
for constructing an Ordered Monotonic Hierarchy [80]. Smith and Peot [123] 
provide a critic of the Alpine algorithm. Lansky and Getoor [87] relate OM 
hierarchies with problem decomposition using constraints. Bergmann and 
Wilke [20] argue for an explicit refinement relations between plans at diflFerent 
levels of abstraction. They also present an enlightening review of their Paris 
abstraction/case-based system in automated manufacturing domain. 

The material of this chapter is largely adapted from an Artificial Intelli- 
gence journal article with Bacchus [13]. The AbTweak system used in the 
empirical test is implemented by Woods and Yang (see [149, 160]). 
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Along with the precondition-elimination type of abstraction, hierarchical 
task-reduction planning^ or HTN planning, is another widely-used hierarchi- 
cal planning method. Given a set of high-level tasks to be carried out, this 
method will find ways for reducing each task into subtasks according to a 
predefined set of task reduction schemata. It then resolves possible conflicts 
among the subtasks in the plan using a least-commitment strategy. The pro- 
cess is repeated until the tasks in the plan are primitive, and all interactions 
between the tasks are removed. The advantage is that a hierarchical planner 
works on a small plan with only an important set of interactions first, before 
it sets out to handle the rest, which are considered as mere details. This tech- 
nique has been used in a number of planning systems, including Noah [114], 
Nonlin [132], and SiPE [147]. 

The domain knowledge of a hierarchical planner is organized hierarchically 
in the form of task-reduction schemata. In addition to a set of primitive tasks, 
a hierarchical planner also has a set of non-primitive tasks, as well as a set 
of task- reduction schemata that defines the relationship between the tasks. 
The efficiency of a hierarchical planner depends on how the task-reduction 
schemata are defined; if one is not careful in the definition, one may lose the 
efficiency of hierarchical planning. 

This chapter presents the following results. 

1. We give a definition for the HTN planning knowledge-base. We specifi- 
cally define how one operator relates to other operators and causal links 
via a reduction mechanism. 

2. We show that with HTN planning, the upward solution property is gen- 
erally not satisfied. This could result in some computational difficulties 
when using the knowledge-base for planning. In particular, if a planner 
faces conflicts in a plan that are impossible to resolve by ordering the 
steps, it cannot abandon the plan right away. Instead it has to try to 
reduce its steps further to see if interleaving the subtasks introduced at 
lower levels could resolve the conflicts. This greatly reduces the efficiency 
of a planner, since it has to search an extra portion of its search space, 
even if there is strong indication that no solution exists in that space. 

3. In response to the above problem, we show that a class of domains exists 
where the task-reduction schemata can be defined in certain ways, so that 
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unresolvable conflicts in a plan cannot be resolved in any reduction of the 
plan. We provide a set of sufficient constraints on schemata definitions 
to ensure this property. 



12.1 Defining Operator Reduction 

Task Reductions 

A hierarchical planner relies on a set of operator reduction schemata In 
some sense, these schemata are similar to the operators O for a single- level, 
total-order or partial-order planner. A reduction schema G ^ is a function 
which, when applied to an operator in O, returns a partially-ordered set of 
operators q;^. R is not necessarily applicable to every operator in (9, and an 
operator can have more than one reduction schema applicable to it. The set 
of operator reduction schemata applicable to a is denoted by RSet(a). a is 
primitive if RSet(a) = 0, otherwise it is non-primitive. Intuitively, a primitive 
operator is one which cannot be decomposed further into more detailed steps, 
while a non-primitive one can. 

If R is applicable to a, then R{a) = (A, B, C, D), where 

— A is a set of operator definitions, 

— B defines the binding constraints on the variables, 

— C is a set of causal links associated with the specification, and 

— D defines a partial ordering among the elements of A. 

R{a) is called a reduction of a. As an example of operator reduction, con- 
sider the household domain. An operator for painting the ceiling and its two 
reductions are given in Figure 12.1. 

For each non-primitive operator a, its reduction schema R{a) defines 
a refinement relationship between a and the set of operators in R{a). Let 
the latter be A{R{a)). The operators in A{R{a)) must obey the following 
restrictions: 

Effect Inheritance. Every effect of an operator is asserted by at least one of 
the operators in its reduction. 

That is, Ve G Eff(a), 3ai G A{R{a)) such that 

— e G EflF(ai), and 

— V/3 G A{R{a)), if -^e G Eff(^) then This condition ensures that 

e is not deleted by some operator in the reduction. 

Precondition Inheritance. Every precondition of an operator is also a precon- 
dition of at least one of the operators in its reduction, and this precon- 
dition is not established by some other operator in the reduction. 

That is, Vp G Pre(a), G A{R{a)) such that 

— p G Pre(ai), and 

— V/? G A{R{a)), if p G Eff(/3) then This condition ensures that 

the precondition p is not internally achieved. 
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paint-ceiling 



reduces to 



R1 (paint-ceiling) 



R2 (paint-ceiling) 



» t 




Fig. 12.1. An example operator reduction schema 



preconds: 

P have(money) 
^ ready(ceiling) 



effects: 



paint-ceiling 



painted(ceiling) 

wet(ceiling) 



^ R1 (paint-ceiling) 



} 




V j 



Pre(buy-paint)={have(money)} 

Pre(buy-brush)={have(money)} 

Pre(brush-paint)={ready(ceiling)} 
Eff(bmsh-paint)={painted(ceiling), wet(ceiling) } 

Fig. 12.2. Restrictions on operator reduction schemas 



Figure 12.2 shows an example where these two restrictions hold. 

In HTN planning we could combine the notions of a goal and a operator 
into a unified notion of a task. In this formalism, a goal is seen as a high-level 
operator to be achieved, and associated with a goal is a set of skeleton plans 
that could be used to achieve the goal. These skeleton plans are themselves 
operator reduction schemata. Figure 12.3 shows an example of goal reduction. 

Likewise, in HTN planning, the plan steps are also called tasks, so that 
the reduction process for a goal, operator and plan step becomes unified. In 
this book, we use the terms “plan steps” and “tasks” interchangeably. 
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Achieve(be-there) 

I 



t 



I 

I 
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Fig. 12.3. Applying a reduction schema to a goal 



Let Si be a plan step in a plan 7J, resulting from instantiating an operator 
a G O by a substitution 6. We extend the definition of operator reduction 
schemata to plan steps as follows: for every reduction function R in RSet(a), 
R{si) = R{a)9^ where R{a)9 is the result of applying the substitution 9 to all 
operators in A{R{a)) (see Chapter 4 for an introduction to substitution), and 
to all the operator occurrences in the causal links of R{a). If R is applicable 
to Si, then R{si) is called a reduction of s^, and a step in R{Si) is called a 
sub step of Si. 

Plan Reductions 

We now consider how to reduce a plan from one level to the next. 

Let s be a step in plan II, and R a reduction schema applicable to s. 
Then R{TI) is the plan which results when s is replaced by R{s) in iJ. What 
remains to be specified is how causal links related to s are transformed in 
R{n), and how order relations are transposed. We call the former causal- 
link-inheritance, and the latter order-inheritance. 

With causal-link inheritance, two cases are to be distinguished. 

Case 1: s is a producer. 

Let 6 = {s ^ st>) be a causal link in II. After 77 is reduced using R, 6 is 
no longer associated with R{II). R{II) will have one or more causal links 
derived from 6 involving a substep Sgub in A{R{s)). In particular, let Sgub be 
a substep of s such that 

1. p is an effect of Sgubj and 

2. p is not negated by any other step after Sgub in R{s). 



By the definition of R, one or more such Ssub always exists. Then {sgub ^ ^b) 
is a causal link associated with R{II). 

Case 2: s is a consumer or user. 

Similarly, let s be a step in a plan 77, and 77 be a reduction schema applicable 
to s. Let 77(77) be the plan which results when s is replaced by 77(s) in 77. 
Let 6 = {Sa ^ s) be a causal link in 77, and let Sgub a substep of s such that 
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1. p is a precondition of Ssubi and 

2. p is not achieved or negated by any other step before Ssub in -R(s). 

By the definition of i?, one or more such Ssub always exists. Then (sa 
is a causal link associated with R{II). 

Order-inheritance, Finally, we specify how precedence relations are trans- 
posed from a plan to its reductions. Our specification here is adapted from 
[ 112 ]. 

Let s be a step in a plan 77, and let Sa be another step such that Sa^s. 
Suppose that s is reduced by R{s). Then in plan IT, we require that in R{II)^ 
step Sa is ordered before the last substeps of s in i?(s). 

Similarly, let be a step in U such that sh^s^. Then in R{II) it is required 
that the first substeps in R{s) be before step S5 in R{II). 

Multi-stage Reductions 

The above definition for one-step plan reduction can be naturally extended 
to reduction in more than one step. Let iJ be a plan, and Ri, R2, , . . ^ Rn be 
reduction schemata. Then 

Qin) = Ri{R2i.,,{Rn{n)).,.) 

is a composite reduction of U. Furthermore, if Zl is a set of causal links asso- 
ciated with 7T, then Q{A) is the corresponding set of causal links associated 
with Q{n). 

A hierarchical planner operates by iteratively reducing the steps in a plan. 
Each iteration performs a number of refinements. Algorithm H REFINE (see 
Table 12.1) depicts how refinement of a plan can be done in this framework. 
In this algorithm, a procedure H Critic takes a plan and applies critics to 
the plan. A critic can be any plan modification operations, including conflict 
resolution, plan merging, step reordering, and variable binding. 

Table 12.1. The HRefine algorithm 



Algorithm HRefine(7T, O, 

Input: A plan II, a set of operators O and reduction schemata 
External Routines: HCRlTlc(Tir) applies a set of critics to il; 
Output: Plan reductions. 

1 . Choose a non-primitive step s to reduce; 

2. SUCC := 0; 

3. for each R G RSet(s) do 

4. Succ := Succ u HCritic(R(7Z')) ; 

5. end for; 

6. return(Succ); 
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Discussion 

Reduction Assumptions. The definition of reduction schemata above is 
intended to capture the formal aspects of operator or goal expansions in a 
number of planning systems. In particular, a reduction schema R corresponds 
to a “soup code” in Noah [114], an “opschema” or an “actschema” in Non- 
LIN [132], and a “plot” in SiPE [147]. However, several simplifications are 
made in our formalization of reduction schemata, the first being that “reduc- 
tion assumptions” are not included explicitly in our definitions. A reduction 
assumption^ [29] is a condition on the applicability of a reduction. For exam- 
ple, to clear the top of a block a?, a planner may choose a particular reduction 
that involves two steps: clear the top of block y that is on top of x, then move 
y to the table. But this reduction is needed only if x is not already clear on 
its top. In this case, On(y, x) is a reduction assumption, and an interval is 
set up that protects the assumption if the reduction is selected. Reduction 
assumptions are the preconditions of certain substeps in a reduction R that 
are not achieved by some other substeps in R, and are used by the control 
component of a planner to restrict the use of certain reductions. The state 
space generated by all possible reductions that include reduction assumptions 
is a subset of the state space defined by our formalization. Since the results 
to be presented below concern the non-existence of solutions in a portion of 
a state space, it will be easy to verify that all of our subsequent results will 
hold with the inclusion of reduction assumptions. Thus, we omit them for 
simplicity. 

Planning Level or Abstraction Level. In planning research, the term 
“hierarchical planning” has been used to describe several different types of 
planning methods, including subgoaling [102], task reduction [114, 132], and 
planning at different levels of details [113, 147]. To clarify the concept, Wilkins 
provides an in-depth discussion of different varieties of planning hierarchies 
[147], and classifies them into either planning levels or abstraction levels. A 
planning level corresponds to the “artifacts of particular planning systems” 
[147]. For example, a new planning level can be created by expanding each 
node in a plan according to some pre-defined task-reduction schemata. On 
the other hand, an abstraction level is distinguished by the “granularity, or 
the fineness of detail, of the discriminations it makes in the world” [147]. 
For example, Abstrips [113] is a planner that plans on different levels of 
abstraction. In this chapter, the term “hierarchy” refers to the planning levels 
that are obtained by reducing tasks in a plan using a pre-defined set of task 
reduction schemata. 



It is also called a “use- when” condition in Nonlin [132] 



1 
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12.2 Upward Solution Property 

A hierarchical planner functions by repeatedly selecting and reducing the 
non-primitive plan steps, and applying critics to modify a plan. A critic is a 
specialized method, either for removing conflicts, or for optimizing the tasks 
in a plan. Sometimes a critic may And conflicts in a plan that are impossible 
to resolve. A plan in this situation is said to have unresolvable conflicts. 
Sometimes, backtracking from such a plan is not the best behavior, since 
although a plan is not correct at the current level of hierarchical planning, 
it may indeed be possible to make it correct once the plan is reduced to the 
next lower level. 

In Chapter 10 we introduced a property known as the upward-solution 
property. It states that the existence of a concrete-level solution implies the 
existence of an abstract level solution. If in a hierarchy, the unresolvable 
conflicts in a plan become resolvable at a lower level, then the upward-solution 
property is clearly lost. In the following, we first show examples of how the 
property can be lost in HTN planning. 



12.2.1 Losing the Property 

We first consider an abstract example. Suppose that a plan contains two 
non-primitive steps a and b, such that between them there are no ordering 
constraints (see Figure 12.4a). Suppose that their effects and preconditions 
are as follows: 

Pre(a) = {i}, Eff(a) = {u,^y}, 

Pre(b) = {y}, Eff(b) = {w,-'x}, 

where the literals u^w^x^y are all distinct. Then the following conflicts will 
occur: an effect of a deletes a precondition of b, and an effect of b deletes 
a precondition of a. This kind of conflict is often called a “double cross,” 
meaning that neither ordering of a and b will work. Unless a planning system 
can resolve this conflict by inserting additional steps between a and b to 
restore the needed preconditions, it will normally announce failure. 

However, suppose that the following reductions are available. 

1. step a can be reduced to the substep ali— >a2, with Pre(al) = Pre(a), 
Eff(a2) = Eff(a), and x ^ Pre(a2), and 

2. step b can be reduced to blH-»b2, with Pre(bl) = Pre(b) and Eff(b2) = 
Eff(b), and y ^ Pre(b2). 

Then the conflict can be resolved in the reduced plan, by constraining the or- 
derings such that alH-»b2 and bl»—^a2 (see Figure 12.4b). Clearly, the upward- 
solution property fails in this hierarchy. 




214 



12. Properties of Task Reduction Hierarchies 



preconds: x 


a 


\ 

effects: u, — i y 


Initial 


preconds: y 

V 


b 


effects: w, — i x 



a 




b 



Fig. 12.4. (a) A plan with unresolvable conflicts, (b) Resolving the conflict by 
reducing the plan in (a), by imposing ordering constraints 



12.2.2 A Variable- Assignment Example 

In the above example, a conflict which appears unresolvable at a higher level 
is in fact resolvable at a lower level. This happened when both high-level 
steps are reduced to consecutive substeps. 

The problem of losing upward-solution property could occur in other types 
of reduction as well. Consider the problem of interchanging the values of two 
variables Figure 12.5. The plan in part (a) contains an unresolvable conflict. If 
we reduce the assign-x step as shown in part (b), however, the conflict could 
become resolvable by interleaving the assign-y step and the two substeps of 
assign-x. 



12.2.3 The Chinese Bear Example 

The above example shows that when a high-level step splits its preconditions 
and effects among different substeps, an unresolvable conflict at a higher level 
could become resolvable. Now we show that this problem could even occur 
when the effects of a high-level step split between different subtasks. 

Consider the following Chinese bear problem. According to a Chinese tale, 
a bear attempted to steal corn from a farmer’s field. It decided to transport 
the corn cobs by placing them under its arms. To begin with, the bear had 
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a 




b 

Fig. 12.5. (a) A variable-assignment plan containing unresolvable conflicts, (b) 
Resolving the conflict by reducing the plan in (a) 



two high-level operators, corn-iinder-larm and corn-under-rainn, referring 
to putting the corn under the left arm and the right arm, respectively. A high- 
level plan consisting of two corresponding steps is shown in Figure 12.6a. This 
plan contains unresolvable conflicts, since in the process of getting an ear of 
corn by its left hand and placing it under the right arm, the bear will lose its 
“treasure” under the left arm, and vice versa. 

In the Chinese tale, the bear spent all night trying to resolve this conflict, 
by repeating the two actions one after another. As day broke, the bear finally 
had to give up and escape into the forest in haste, empty handed. And, as 
to the farmer, he woke up with a pleasant surprise that the entire corn field 
had been neatly harvested! 

Our bear could of course do better by simply refining the plan to the next 
level down, and by interleaving the sub-steps. This is shown in Figure 12.6b. 
In this plan the bear could indeed put both ears of corn under its arms; it 
could grab both ears of corn first, one in each hand, then place them under the 
opposite arms in any order. Using step reduction, the bear could successfully 
get both corn cobs home. 
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a effects: 




I 

Y 



effects: 




effects: 

have(Cornl) 



Fig. 12.6. (a) A Chinese bear plan with unresolvable conflicts, (b) Resolving the 
conflicts by reducing the plan in (a) 

12.3 Imposing Restrictions 

12.3.1 Motivation 

From the search-control point of view, we would like the upward-solution 
property to hold. If we know that, for a given domain, the operator-reduction 
schemata satisfy this property, then we could backtrack from the current plan 
whenever the plan contains unresolvable conflicts. However, if the reduction 
schemata do not satisfy the property, the planner cannot rule out reducing 
the plan steps further as a means of resolving the conflicts. 

One way to guarantee the upward-solution property is to impose restric- 
tions on how the reduction schemata are defined. We would like the restric- 
tions to be syntactic in nature, so that checking them against any user-defined 
reduction schemata could be done automatically. These restrictions may not 
be satisfied in all domains, but in some cases we could use them to preprocess 
the set of operator reduction schema by redefining the reduction relations. 
We could also identify the subset of operators which satisfy the restrictions 
a priori. 

In the following sections, we will consider one such restriction. 
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12.3.2 Unresolvable Conflicts 

Let iT be a plan. Let C-Links(il) be the causal links in the plan. We say 
that n has unresolvable conflicts if in all total orders of the plan steps, some 
causal links are violated. That is, in every total order of the plan steps, for 
some causal link (s^ sj) there exists a plan step s^, such that 

— s/b is before Sj and after Si in the total order, and 

— -ip is an effect of Sfc. 

Let iJ be a plan with unresolvable conflicts. The upward-solution property 
is equivalent to the following statement: let # be a set of reduction schemata. 
If Q{n) is any composite reduction of il using the reduction schemata in #, 
then Q{n) also contains unresolvable conflicts. 

If this property is satisfled by a set of reduction schemata for every plan 
and every composite reduction of the plan, then we say that ^ is downward- 
unresolvable. 

12.3.3 The Unique-Main- Subaction Restriction 

We now propose a simple restriction for ensuring the property. The restriction 
requires that every non-primitive operator has a unique main sub-operator, 
or subaction, that inherits all preconditions and effects of the parent. 

Definition 12.3.1 (Unique Main Subaction Restriction). Let a be an 

operator and R be a reduction schema applicable to a. The operators in 
A{R{a)) satisfy the following conditions: 6 A{R{a)) such that 

— am asserts all effects of a, and no other sub- operators of a assert any effect 
of a. 

In other words, Eff(o:) C Eff(am) and^ai € A{R{a)), such that ai / am, 
Eff(o;i) n Eff(a) = 0. 

— am requires all preconditions of a, and none of these conditions is achieved 
by any other sub-operators of a. 

In other words, Pre{a) C Pre{am), and^ai G A{R{a)), such thatai ^ am, 
if ai^am then Eff(ai) H Pre{a) = 0. 

This restriction will be referred to as the Unique- Main- Subaction Restric- 
tion^. Intuitively, it requires that, for every step s in a plan 77, its reduction 
R{s) contains a “main” substep s^, which asserts everything s asserts. In 
addition, the preconditions of s are required to “persist” till the beginning of 
Sm • 

A number of planning domains can be represented in ways that satisfy this 
restriction. For example, consider an operator of fetching an object “object 1” 

^ Although for consistency reasons we should call it unique-main-sub- operator re- 
striction, we nevertheless choose to follow its original term from [152]. 
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in a room “rooml” (based on [147]). The operator ‘Tetch” can be reduced into 
a sequence of three substeps: getting into rooml, getting near object 1, and 
picking up object 1, in that order. The third substep, picking up object 1, can 
be considered as the main substep in this reduction. Notice that the condition 
that objectl is in rooml is a precondition of both the non-primitive operator 
“fetch” and the main substep. The complete reduction schema is given in 
Table 12.2. 

Let O be the set of operators of a planning system, and ^ be the set 
of operator-reduction schemata. We say that # satisfies the Unique Main 
Subaction Restriction if and only if for every reduction schema R G ^ and 
every operator a G O to which R is applicable, R{a) satisfies the Unique 
Main Subaction Restriction. The following theorem can be easily proved by 
induction on the number of steps inserted into a plan. 

Theorem 12.3.1. Every set of operator-reduction schemata satisfying the 
Unique Main Subaction Restriction is downward-unresolvable. 

Before discussing how this theorem can be used for checking the down- 
ward-unresolvable property, we would like to comment on the applicability of 
the Unique Main Subaction Restriction to planning domains. Domains that 
satisfy this restriction have the following characteristics: 

1. The goals to be achieved can always be broken down to several less com- 
plicated subgoals to solve. For each subgoal, a number of primitive oper- 
ators are available for achieving it. Each such operator requires several 
preparation steps before it can be performed, and a number of clean-up 
steps after it is done. This operator can be considered as the main step 
in a reduction schema. 

2. For each group of operators mentioned above, a hierarchy is built by 
associating with a non-primitive operator a set of effects which are the 
purposes of the main step mentioned above, and a set of preconditions 
which are certain important preconditions of the main step. 

A number of domains can be formulated in ways that satisfy the property 
of downward- unresolvability. Consider again the painting domain. Recall that 
in this domain there are a room, a ladder, supplies for painting as well as a 
robot whose goal is to paint portions of the room and/or the ladder. Suppose 
in addition to the operators and reduction schemata for painting, the robot 
also knows how to fetch an object using the operator fetch in Table 12.2. 
Represented in this way, every operator-reduction schema satisfies the Unique 
Main Subaction Restriction. For instance, the apply-paint -to-ceiling op- 
erator in the reduction of paint (Ceiling) is the main sub-operator. Also, the 
pickup step in the reduction of fetch is also a main sub-operator. Moreover, 
these reduction schemata all satisfy the Unique Main Subaction Restriction. 




12.4 Preprocessing 219 



Table 12.2. An operator-reduction schema satisfying the Unique Main Subaction 
Restriction 



preconditions 


effects 


fetch 

Inroom(Objectl, Rooml) 


Holding(Robotl, Objectl) 


achieve(Inroom(Robotl, Rooml)) 
0 


Inroom(Robotl, Rooml) 


achieve(Nextto(Robotl, Objectl)) 
0 


Nextto(Robotl, Objectl) 


pickup 

Inroom (Objectl, Rooml), 
Inroom ( Robot 1, Rooml), 


Holding(Robotl, Objectl) 


Nextto(Robotl, Objectl) 





12.4 Preprocessing 

One advantage of the Unique Main Subaction Restriction is that it allows for 
efficient preprocessing of a given set of reduction schemata. Specifically, this 
restriction only limits the way a non-primitive operator relates to its set of 
sub-operators. Thus the checking process could proceed from an operator to 
its direct descendents, in an orderly fashion. 

To check if R{a) satisfies the Unique Main Subaction Restriction, a test is 
first made to see if there is a unique main sub-operator in A{R{a)) that asserts 
every effect a asserts and has in its set of preconditions every precondition of 
a. If so a subsequent check is made on every other sub-operator in A{R{a)), 
to make sure that no other sub-operator of a asserts a precondition or an 
effect of a. 

For each operator and each reduction schema R, verifying the Unique 
Main Subaction Restriction requires a time complexity of 0{\0\), where \0\ 
is the number of operators in O. Let k be the maximum number of operator 
reduction schemata applicable to an operator. Then UmsCheck has a total 
worst-case complexity of 0{k x |Op). 

If the UmsCheck returns true, then the set of reduction schemata 
satisfies the downward-unresolvability restriction. In that case, whenever a 
hierarchical planner detects unresolvable conflicts in a plan, it does not have 
to consider the reduction of the plan as a means of resolving the conflicts. 



12.5 Modifying Operator- Reduct ion Schemata 

Even if UmsCheck returns false for in some cases it might still be 
possible to modify the schemata, so that after the modification, 0 satisfies 
the restriction. 
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Table 12.3. The UmsCheck algorithm 



Algorithm UmsCheck(C1, 

Input: A set of operators O and reduction schemata 

Output: TRUE iff the unique-main-subaction restriction is satisfied. 

1. B := O' 

2. while jB 7 ^ 0 do 

3. a := Remove- element{B); 

4. if VR G RSet(a), 

5. R{a) satisfies the Unique-Main-Subaction Restriction then 

6. Mark a; 

7. end if 

8. B := B — {q}; 

9. end while 

10. if all operators in O are marked then 

11. return(TRUE); 

12. else 

13. return(FALSE); 

14. end if 



We illustrate the modification process via Figure 12.7. In the figure, a 
is an operator with a precondition p and an effect q. Suppose R{a) consists 
of two sub-operators al followed by a2, such that the precondition of al 
is p, and the effect of a2 is q (see Figure 12.7a). Clearly, the Unique Main 
Subaction Restriction is violated. One way to modify this reduction schema is 
to remove the precondition p from Pre(a). As a result, the modified reduction 
i7' satisfies Unique-Main-Subaction Restriction. This new reduction schema 
is shown in Figure 12.7b. 

The above example shows how to modify a reduction schema by removing 
the preconditions of an operator. Other methods for schemata-modification 
can also be designed. For instance, let a be an operator with two effects q and 
u such that a sub-operator al of a has an effect q while another sub-operator 
a2 has an effect u (see Figure 12.8a). If u is considered as a side-effect of a, 
and is considered to be less important than q^ then removing u from the the 
effects of a will make the reduction schema satisfy the Unique Main Subaction 
Restriction. This is shown in Figure 12.8b. 

Unfortunately, the above examples do not suggest any general procedure 
for modifying a given set of operator-reduction schemata. This is because 
changing the representation of the operators in O may make them less ex- 
pressive than before. This can occur when, for example, some effects of an 
operator are removed from its effects set. In extreme cases, changing the rep- 
resentation of an operator may cause planning to be less efficient than before. 
For example, removing the preconditions from an operator may delay the de- 
tection of deleted-condition conflicts with the removed conditions, and as a 
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b 

Fig. 12.7. Modifying a reduction schema (a) by removing some preconditions of 
an operator (b) 

result, more effort in reasoning may be needed to fix those conflicts after the 
modification. On the other hand, there may exist classes of planning prob- 
lems for which a particular kind of schemata modification method exists, and 
finding out the characteristics of these domains would be a very interesting 
exercise. 



12.6 Open Problems 

More Expressive Languages 

So far we have considered hierarchical planning with a particular kind of oper- 
ator representation, in which the preconditions and effects of an operator are 
literals. This restricts the techniques developed to be only useful for a subset 
of the existing hierarchical planning systems. For example, SiPE [147] allows 
for conditions which can be context-dependent, quantified, and disjunctive. 
In addition, some effects can be deduced from a set of external axioms. With 
operator representations as expressive as these, some of our results no longer 
apply. 
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a 




b 

Fig. 12.8. Modifying a reduction schema (a) by removing effects of operator a (b) 



For example, consider operators with preconditions in the form of disjunc- 
tions of conditions. Then the effects of a group of operators can collectively 
deny or assert the preconditions of other operators. With these representa- 
tions, the detection of conflicts during planning is a more complicated prob- 
lem. It is believed that the idea of extracting information about conflict a 
priori can be used to help ease this computational burden, in the same way 
this chapter has shown. For example, information about which operators are 
more likely to interact may be obtained through preprocessing and used to 
provide a better control strategy for the detection and handling of conflicts. 

Heuristics for Planning 

Preprocessing can also provide a planner with good heuristics for choosing 
among alternative operator-reduction schemata. To reduce a non-primitive 
operator, there is usually more than one available schema, and current hi- 
erarchical planning systems select the alternatives in a depth-first manner. 
Depending on which reductions are chosen, different conflicts may be intro- 
duced, and this may affect the planning efficiency. 
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One heuristic which can be used in choosing a reduction is to minimize the 
number of conflicts introduced. Preprocessing can be helpful for predicting 
which reductions are likely to yield a set of conflicts that is small — and just 
as importantly, a set of conflicts which are easy to resolve. 



12.7 Summary 

In this chapter we have outlined a formalization of hierarchical planning under 
task reduction. Based on the formalism we have also developed a syntactic 
restriction on the relationship between a non-primitive operator and its set of 
sub-operators. When satisfled, the restriction enables hierarchical planning 
systems to recognize a dead-end state earlier. Conflicts in a plan can then be 
concluded as unresol vable at every level of reduction below, given that they 
are unresolvable at the current level. Because of the syntactic nature of the 
restrictions, it is possible to preprocess a given set of reduction schemata. 



12.8 Background 

The basic structure of hierarchical-task network planning was determined 
by two influential systems, Sacerdoti’s Noah [114] and Tate’s Nonlin [132]. 
Wilkins gave a comprehensive overview in [147]. 

There is an increasing amount of interest in HTN planning, particularly 
due to its ability to effectively encode much knowledge about both problem 
decomposition and hierarchical abstraction. On-going investigations include 
papers appearing in AIPS94 [161] and AAAI94 ([18], [43], and [133]). 

The material of this chapter is adapted from a Computational Intelligence 
journal article [152], which in turn was an adaptation of part of Yang’s PhD 
thesis [150]. Nau and Hendler provided much valuable advice to the latter. 
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In this chapter we are interested in identifying the primary effects of planning 
operators for the purpose of improving planning efficiency and maintaining 
solution quality. In many realistic domains, a planning operator might have 
a large number of effects; only a few of which are relevant to the problem at 
hand. The idea of planning with primary eflFects is to specify certain effects 
of planning operators as primary^ and to choose operators for achieving goals 
only in terms of the operators’ primary effects. Identifying primary effects 
among operators is analogous to building a two-level abstraction hierarchy 
on the effects of operators. The primary effects are on the top level. During 
plan synthesis, only the top-level effects are considered for achieving subgoals. 
Consideration of all effects is done only when plan correctness is verified. 

This chapter presents a formal model of planning and learning with pri- 
mary effects. 



13.1 A Motivating Example 

The intuition behind primary effects is quite natural. Consider the household 
domain for example. A painting operator might have many different effects, 
as shown in Figure 13.1, ranging from Painted(Wall) to Exercised( Agent). 
Among those effects, we would normally consider Painted(Wall) as a primary 
effect of painting. The other effects, such as Exercised( Agent), are considered 
secondary. Put it another way, if an agent only wants to exercise, he or she 
could choose a much cheaper and more pleasant way of doing it, by jogging. 



paint(wall) 



effects: 

Painted(Wall) 

Empty(Painttray) 

Wet(Brush) 

Sticky (Wall) 

Exercised(Agent) 

Small(Room) 



Fig. 13.1. A painting operator 
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riding a bike, etc. Painting is invoked only when Painted (Wall) is posed as 
a goal to achieve. As another example, the primary eflFect of breaking a hole 
in an office wall is to create a door or a window. If we merely wanted to go 
to another office, we could do it more easily by simply walking to that office 
through a doorway. 

Primary effects have been used in planning in a fairly informal way in 
several planning systems. 

— Strips [46] utilizes primary effects as a means to control the quality of 
solution plans. 

— SiPE [147] distinguishes between the “main effects” and the “side effects” 
of operators and uses this distinction to simplify conflict resolution in plan- 
ning. 

— Prodigy [25, 80] allows the user to specify primary effects of operators, 
which are then used as an effective method to control search. 

— AbTweak [160] also allows the user to specify primary effects, the use of 
which significantly improves the efficiency of planning. 

In all these systems human designers are relied upon to identify the primary 
effects, and no effort has been made to build a well-founded guidance for the 
selection of those effects. 

In this chapter, we address several important issues involved in under- 
standing primary effects. First, we present a representative algorithm for 
planning with primary effects. Then we analyze the effectiveness of using pri- 
mary effects in improving search efficiency and solution quality. Finally, we 
present a learning algorithm for automatically identifying the primary effects 
of operators and analyze its effectiveness. 



13.2 Primary-Effect Restricted Planning 

Given that primary effects of planning operators have been chosen, it is in 
fact very easy to modify a partial-order planner to obtain one that uses only 
primary effects to achieve subgoals. To see this, let POPLAN be a backward- 
chaining, partial-order planner that does not distinguish between primary 
and secondary effects, and let PrimPop be its primary-effect counterpart. 
We will call PrimPop a primary- effect restricted planner, and the original 
PoPLAN planner an unrestricted planner. 

In most implementations of partial-order planners, a key component is 
how to select operators to achieve subgoals. These steps are shown charac- 
teristically in Table 13.1a. To modify the planning algorithm so that only 
primary effects are considered for achieving subgoals, we need to modify only 
a few lines, as shown in Table 13.1b. 

A major purpose of using primary effects in planning is to restrict the 
number of possibilities when achieving a subgoal. In terms of search, this 
corresponds to reducing the branching factor of the search tree. 
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Table 13.1. Comparing unrestricted and primary-effect restricted planners. The 
differences between the two planners are highhghted by boldface 



Algorithm Poplan(77, O , . . .); 
Input: 

An initial plan 77, 

a set of planning operators O 

etc. 

Output: 

A correct plan if there is one; 

Fail otherwise. 



/* achieve new subgoals */ 
choose a subgoal (p,s) from 77, 

where p is an open precondition of s; 

a) find all operators a in O 
such that an effect of 

a unifies with p; 

b) find all existing steps Si in FI 
such that an effect of 

Si unifies with p, and 
that Si can be ordered before s; 
for each such operator a 
or step Si do 

update plan 77, resulting in 
a successor plan; 



Algorithm PrimPop(77, O , . . .); 
Input: 

An initial plan 77, 
a set of planning operators (9, 
with chosen primary effects, etc. 
Output: 

A correct plan if there is one; 

Fail otherwise. 



/* achieve new subgoals */ 
choose a subgoal (p, s) from 77, 

where p is an open precondition of s; 

a) find all operators a in O 
such that a primary effect of 
a unifies with p; 

b) find all existing steps Si in 77 
such that a primary effect of 
Si unifies with p, and 

that Si can be ordered before s; 
for each such operator a 
or step Si do 

update plan 77, resulting in 
a successor plan; 



a 



b 



As an example, consider an office domain with an agent and several ob- 
jects that can be moved from room to room. There are several operators in 
this domain; a few of them are depicted in Table 13.2. An agent can move 
around offices by simply walking from one room to another. However, there 
are also other ways of moving the agent around — by carrying a book, or 
by pushing a trolley. In order to just go to another room, the simplest way 
is to walk without carrying a book or using a trolley. From this intuition, an 
assignment of primary effects as shown by *’s in Table 13.2 seems natural. 

An example search tree for achieving the goal of Inroom (Agent, Room2) 
is shown in Figure 13.2. Part (a) shows a search tree of an unrestricted plan- 
ner, where all effects of all operators are considered for achieving a subgoal. 
Part (b) shows a tree corresponding to a primary-effect restricted planner. 
This tree is considerably narrower because the nodes at each level of the tree 
are a subset of the nodes at a corresponding level of the tree in part (a). 
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Table 13.2. Operator definition, with selected primary effects, for the office domain 
example 



Precondition 


Effect 


Cost 


walk(Agent, Rooml, Room2) 


Handempty, 

Inroom( Agent, Rooml), 
Open (Door) 


* Inroom( Agent, Room2), 
-iInroom( Agent, Rooml) 


1.0 


carry (Agent, Book, Rooml, Room2) 


Holding( Agent, Book), 
Inroom(Book, Rooml), 
Inroom (Agent, Rooml) 
Open(Door) 


* 

Inroom(Book, Room2), 
-iInroom(Book, Rooml), 
Inroom (Agent, Room2), 
-iInroom( Agent, Rooml) 


5.0 


push-trolley (Agent, Trolley, Book, Rooml, Room2) 


Ontrolley (Book) , 
Inroom(Book, Rooml), 
Inroom (Agent, Rooml), 
Inroom (Trolley, Rooml), 
Open(Door) 


Inroom(Book, Room2), 
-tInroom(Book, Rooml) 
Inroom( Agent, Room2), 
-iInroom(Agent, Rooml) 
*Inroom(Trolley, Room2), 
♦-ilnroom (Trolley, Rooml) 


10.0 




Fig. 13.2. Search trees of unrestricted planning (a) and primary-effect restricted 
planning (b) 
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This example shows a general fact about using primary effects: primary- 
effect restricted planners usually have smaller branching factors. This implies 
that, if we somehow limit the depth of the tree, a primary-effect restricted 
planner could expand a smaller search tree, and is thus more efficient than 
an unrestricted planner. 

The same example shows another point about using primary effects. Sup- 
pose that one uses a depth-first search control strategy to operate an un- 
restricted planner. The planning process then corresponds to searching the 
tree in Figure 13.2a in a top-to-bottom order. If the first branch involving 
a trolley actually leads to a solution, it is conceivable that this solution is 
much more costly than the one which involves walking only. On the other 
hand, if a primary-effect restricted planner is used, a tree similar to the one 
in Figure 13.2b is searched. The first solution, again in a top-to-bottom order, 
is simply the optimal solution in this case. Thus, in this case, using primary 
effects also improves the solution quality. This is the main motivation behind 
Strips in its use of primary effects. 

In general, proper uses of primary effects in planning can lead to higher 
planning efficiency and improved solution quality. This conclusion, however, 
corresponds to a best-case scenario; it assumes that the primary effects are 
chosen appropriately. In the next section, we show that an arbitrary selection 
of primary effects could lead to an erosion of planning efficiency as well as 
solution quality. 



13.3 Incompleteness and Sub-optimal Solutions 

Using primary effects in planning corresponds to cutting down the number of 
possible plans to achieve a given goal. This is because by restricting a plan- 
ner’s attention only towards the primary effects, all operator sequences that 
achieve a goal based on the side effects of the operators will not be part of the 
search tree of the planner. The implication is that if primary effects are chosen 
poorly, a primary-effect restricted planner may not be able to find a good so- 
lution to a planning problem and, in some cases, may not find a solution at all. 

As an example, in the planning problem shown in Figure 13.3, suppose 
that each square represents a plan step, and every path in the directed graph 
represents a solution to the planning problem. The crosses represent those 
branches that are considered by an unrestricted planner, but are not consid- 
ered by a primary-effect restricted planner. If the number of steps (that is, 
squares) in a path, from start to finish, represents the cost of a plan, then the 
particular selection of primary effects which gives rise to the remaining path 
in the graph leaves a longer, sub-optimal solution plan to the planning prob- 
lem. In this example, we see clearly that an arbitrary selection of primary 
effects might lead to sub-optimality. 

In extreme cases, a poor selection of primary effects could also lead to 
incompleteness of planning. In the household example, if painting is the only 
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Fig. 13.3. Demonstrating problems of primary-effect restricted planning 



means for the agent to do exercise, and if the effect Exercised (Agent) is not 
chosen as a primary effect of operator paint, then the agent will have no 
solution plan for doing exercise. 

We summarize these two problems as follows: 

Incompleteness. Inability to find a solution to a solvable planning problem. 
Sub-Optimality. Even if a solution plan can be found, a primary-effect 
restricted planner might only find a poor solution to a planning problem. 

In the next section, we will describe an approach to solving these two 
problems. Our solution will be to use a learning algorithm, which will per- 
form a series of training exercises to obtain a good bias on its primary effect 
selection. 



13.4 An Inductive Learning Algorithm 

Starting with a set of operators for which no primary effects have been se- 
lected, we would like to design a learning algorithm for constructing a selec- 
tion of primary effects based on a set of training examples. Each example is 
a planning problem typically seen by the planner, and the set of examples 
could be built based on the past experiences or on randomly generated train- 
ing sets. After learning, we would like the learned primary effects to have 
good quality. Specifically, we would like to approach a selection of primary 
effects that ensures reduced planning time and near-optimality in solution 
quality, on average. 

The intuition behind the learning algorithm is as follows. For each plan- 
ning problem, consisting of a start-step/finish-step pair, we run an optimal, 
unrestricted planner and a primary-effect restricted planner side by side. If 
and when both planners are finished, we then compare their solution qual- 
ity. Let ilo be the optimal plan returned by the unrestricted planner, and 
Up a plan returned by the primary-effect restricted planner. Then we check 
whether the following conditions are satisfied: 



Cost(ilp) < C * Cost(iJo) 



(13.1) 
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In the above equation, the factor C is a user-defined cost-increase value, 
representing how much the user is willing to tolerate a degradation in solution 
quality; a solution Tip is near-optimal if its cost is a C-factor to the optimal 
cost. For instance, when C = 3 it means that the user is unwilling to have 
a primary-effect restricted planner return solutions that are more than three 
times as costly as the optimal solutions. 

If (13.1) is satisfied, then the learning algorithm will go on to test the next 
problem. Otherwise, the learning algorithm will convert additional effects 
of operators to primary. The rationale for this operation can be informally 
explained as follows. Suppose that condition (13.1) is violated. This could 
be due to a higher cost in the solution plan of the primary-effect restricted 
planner. Now recall that the branching factor of a primary-effect restricted 
planner is generally smaller than that of its unrestricted counterpart, whereas 
the search depth of the former is generally larger than that of the latter. 
Converting effects into new primary effects will have the impact of increasing 
the search space size for a primary-effect restricted planner. The solution 
quality in a larger space will be no worse than that in the space before the 
conversion. Therefore, condition (13.1) will be satisfied by more samples the 
next time around. 

We now describe the algorithm formally. The LearnPrim algorithm, 
shown in Table 13.3, takes a randomly generated problem in each iteration. 
It then computes a plan using an optimal unrestricted planner OptimalPop 
and then using a primary-effect restricted planner PrimPop. If both plan- 
ners terminate within a reasonable time bounds, the algorithm verifies condi- 
tion (13.1) using the accumulated time and cost information. In the event the 
condition fails, LearnPrim calls a subroutine UpdatePrim, which turns all 
effects in the causal links of Uq to primary effects, for operators that corre- 
spond to the steps producing these effects. The LearnPrim algorithm stops 
learning when a certain termination condition TERMINATE holds. In the next 
section, we derive this termination condition. 

We illustrate the operation of this algorithm through the robot exam- 
ple shown in Table 13.2. Suppose that initially no primary effect is chosen. 
Suppose also that the user has provided a set of two training examples: 

( init, Gi ), { init, G 2 ) 

where init is an initial state in which both the agent and a book are in Rooml, 
G\ = {Inroom( Agent, Room2)} and G 2 = {Inroom(Book, Room2)}. Also, let 
C = 2. 

For the first problem ( init, Gi ), an optimal solution found by POPLAN 
is 

n\ = (sin 2 i^walk(Agent, Rooml, Room2)\-->s finish) 

Since at this point no primary effects have been chosen, PrimPop will an- 
nounce failure. In this case, Up has a cost of infinity, and condition (13.1) 
fails. UpdatePrim will then examine the causal links of II i, and find out 
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Table 13.3. The LearnPrim algorithm 



Algorithm LearnPrim(O) 

Input: A set of planning operators O. 

External Function: 

NextProb() returns an initial plan consisting of a randomly generated start-finish 
step pair. 

Terminate() is a boolean function. See the next section for its specification. 
Output: O with selected primary effects. 



1. loop 

'^init NextProb(); 

3. Uo := OptimalPop(77|j^j|.); 

4. 77p := PRiMPop(77jnjt); 

5. if Condition 13.1 is not TRUE, then 

6. UpdatePrim(/7o, O) 

7. end if; 

8. until Terminate()=true; 

9. return((9); 



Subroutine UpdatePrim(77o, O) 

1. for each ((s^ Sj)) G C-Links(/7o) do 

2. let a be the operator in O corresponding to s^; 

3. make p a primary effect of a; 

4. end for 



that the plan step walk produces the precondition Inroom(Agent,Room2) for 
s finish’ This effect of walk will be a new primary effect. 

Continuing with the second planning problem in the same manner, the 
effect Inroom(Book, Room2) will become a primary effect of operator carry. 
Suppose that these two problems are the only examples supplied by the user, 
then at some point (see the next section), the learning algorithm would ter- 
minate. The end result is shown in Table 13.2. 

This example shows another interesting feature of the learning algorithm. 
By learning primary effects, certain redundant operators could be identified 
and removed from the operator set O. In the robot example the redundant 
operator is push-trolley. Removing these operators could further improve 
the efficiency of planning. 
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13.5 How Many Training Examples 

At this point, it has not been specified how the examples are generated, and 
how many examples need be used to train the planner. We answer these 
questions below. 

13.5.1 Informal Description 

The LearnPrim algorithm depends on a routine NextProb() for generating 
the next problem instance. A problem instance is an initial state and goal pair. 
While these problems are generated randomly, an important requirement is 
that the probability distribution of the examples used in the learning phase 
must be the same as the distribution of the problems to be solved later by 
the primary-effect restricted planner, using the selected primary effects. To 
obtain the distribution, we can use a pool of existing plans that are solved 
by an unrestricted planner in the past. Each of these plans is associated with 
a cost, an initial state, and a goal. The relative frequency of each problem 
instance could then be used to approximate the probability distribution. 

Another method for the generation of training examples is to rely on a 
query generator. The query generator is a potential user of the primary-effect 
restricted planner, and will pose planning problems to be solved at a given, 
fixed-probability distribution. As an example, a frequent database user could 
be considered as a query generator; so is a typical client of an information 
provider. 

Suppose that we fix a method for implementing NextProb(). We now 
consider the second question — how many examples are needed. Informally, 
each example (Initial, Goal) is a point in a space of problem instances (see 
Figure 13.4). After an iteration of LearnPrim, the example (Initial, Goal) is 
considered learned. That is, the next time we run PrimPop on the same ex- 
ample, condition (13.1) will be satisfied. In addition, due to the newly selected 
primary effects, a few other problem instances that are near (Initial, Goal) 
could potentially be covered as well. Thus, each training example has a cer- 
tain coverage of learned examples in the space of all problem instances. This 
is shown as a circle around the (Initial, Goal) in Figure 13.4. In answering 
how many examples are needed, it is hoped that, with enough examples, most 
of the area where a problem occurs with high probability will eventually be 
covered. 

In fact, many of the intuitions about the problem coverage have been 
addressed in a formal learning model known as PA G Learning. In the follow- 
ing, we briefly review the main points in this computational model, and then 
apply the model to our problem at hand. 

13.5.2 A Brief Introduction to PAC Learning 

“PAC learning” is a shorthand for probably approximately correct learning. It 
is an inductive learning method for hypothesis selection. Let X be a set of 
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coverage 



a training example (Initial, Goal) Problem Space 




occurring 

problems 



Fig. 13.4. Informal description of the learning process 



all possible examples. X is known as the instance space. A concept over X 
is a subset of X, representing some features satisfying a certain condition. A 
concept class H over X is the set of all concepts on X; this is known as the 
hypothesis space on X. Each element h of is a hypothesis. A hypothesis 
is similar to a predicate: for each x G X, h{x) = TRUE if and only if x is a 
member of the subset of X defined by h. 

Let D be an possibly unknown probability distribution on the instance 
space X. Every element x in X has a probability of occurrence, Pv]j{x). For 
a subset 5 of X, the probability for S to occur is 

^Prp(i) 

xES 

Let t be a target concept to be learned. The elements x of X are classified 
by t into positive examples, for which t{x) = TRUE, and negative examples 
for which t{x) = FALSE. 

It is sometimes impossible to learn the exact target concept t. So we might 
settle for an approximate concept or hypothesis h, such that the error in h is 
small. With the above definitions, the error in a hypothesis could be defined 
formally: 

error(/i) = Prp{x|/i(x) ^ t{x)} 

A hypothesis is approximately correct if error (h) < e for some user-defined 
small and positive real- value e. 

The goal of PAC learning is to learn an approximate concept by sampling 
the elements of X according to the distribution D, for a finite number of 
times. Each training example x is supplied with a correct answer, t(x). For 
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a given sequence of training examples, a hypothesis h is called consistent if 
for all training examples x in the sequence, h{x) = t{x). 

The training samples are taken randomly, so there is no absolute guaran- 
tee that the examples seen by a learning algorithm will be typical enough to 
be useful. In this case, we settle for a probably approximate correct concept, 
in the following sense. Let 6 be small and positive real value. We wish to 
guarantee that, after enough samples are used in training, with probability 
no less than 1 — (5, the concept h we learn is approximately correct. 

Let m be the number of examples used to learn a probably approximately 
correct concept h. We wish m to be finite and reasonably bounded. There is 
a fairly simple but very powerful result in PAC learning, which states that 
if the hypothesis space H is bounded, then for any sufficiently large m, any 
consistent concept /i is a PAC concept. 

We derive this result. Suppose that h' is a bad hypothesis; error{h') > e. 
Suppose that h' is consistent with all m examples. The probability that h' 
agrees with any individual example is less than (1 — e), and the probability 
that h' will be consistent with all m examples is less than (1 — e)"^. The 
probability for a bad hypothesis to be consistent is |^|(1 — e)"^. For this to 
be less than S, 

\H\{l-e)^ <6 

This formula gives rise to an upper bound on m: 

m > -(In i 4- In |iJ|) 

6 0 

Thus, a hypothesis h is PAC if it is consistent with m examples, where m is 
given as above. 

13.5.3 Application of the PAC Model 

We now apply the PAC model to analyze the number of examples needed for 
LearnPrim to learn a good selection of primary effects. We first instantiate 
the PAC model in terms of our primary effect learning problem. 

The primary-effect learning problem could be modeled as follows. An 
example instance is a pair consisting of an initial state and a goal condition. 
The instance space is the space of all (Initial, Goal) pairs. A concept, or 
a hypothesis, is a selection P of primary effects on operators O. P gives 
rise to a set of {init, goal) pairs for which equation (13.1) is satisfied. The 
hypothesis space consists of all possible selections of primary effects. Let \E\ 
be the total number of operator/effect pairs. The size of the hypothesis space 
is \H\ =21^1. 

A selection P of primary effects is consistent with a problem instance 
(Initial, Goal) if a solution can be found by PrimPop such that condi- 
tion (13.1) is satisfied. Given this mapping, we know that any consistent 
selection of primary effects satisfies the PAC property as long as the number 
of training examples is determined by the following formula: 
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i(ln 1 + ln2l^l) = i(ln i + |i;|ln2) (13.2) 

e 0 € 0 

In the above formula, it is assumed that the small positive values e and 6 are 
given. This formula forms a basis for an implementation of the termination 
condition Terminate(), for the LearnPrim algorithm; a counter will be 
incremented until its value exceeds the one in equation (13.2). 

What remains to be verified is whether the LearnPrim algorithm returns 
a consistent hypothesis. This condition is needed for the PAC argument to 
follow. Recall that subroutine UpdatePrim converts eflFects into primary 
eflFects based on the causal links in a solution plan Up. After this is done, 
the next time PrimPop is called to solve the same planning problem, condi- 
tion (13.1) must be satisfied, since this time around PrimPop will return a 
plan of the same quality as the optimal solution. Ho- Applying this argument 
to every example problem used in learning, we know that when the algorithm 
terminates, the selected primary effects must be consistent with all examples 
used in learning. Therefore, when it terminates, the algorithm LearnPrim 
returns a selection of primary effects satisfying the PAC property. 

As an example of the learning model, consider a domain with ten operators 
and two effects for each operator. This domain gives a total of E' = 20 
operator-effect pairs in the domain. If we would like to guarantee that, with 
a probability higher than 0.95 (that is, 6 = 0.05), our selection of primary 
effects to solve 90% of the planning problems that typically occur, that is, e = 
0.1, we could present the learning algorithm with m < 10(ln(l/0.05)4-201n2) 
examples. In other words, LearnPrim should stop when it sees 168 examples. 



13.6 Open Problems 

Applying Palo to Learning Primary Effects 

The Palo algorithm by Cohen, Greiner, and Schuurmans [32] provides an 
efficient approach to inductively learning theories from given samples. An 
application of that system to the primary-effect learning problem is an inter- 
esting approach. For example, using the Palo system it might be possible for 
the learning algorithm to terminate earlier than indicated by equation (13.2), 
which in turn makes it possible to select fewer primary effects than the algo- 
rithm we presented. 

Problem Distribution 

A good selection of primary effects clearly depends on a set of the frequently 
occurring problems. The problem distribution on this set, however, could 
change with time. During a regular teaching term a professor might spend 
more effort solving problems related to teaching and examination marking. In 
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the summer, however, the same person might encounter more travel-related 
planning problems. The primary-eflFect learning algorithm works with a fixed, 
although unknown, distribution of problems. It would be useful for a planner 
to adjust its primary-effect selection with a changing distribution. 

Problem-Dependent Primary Eflfects 

Depending on the particular problem at hand, the same effect of an op- 
erator might or might not be considered as a primary effect. Our algorithm 
LearnPrim could be applied to different sets of training examples and there- 
fore result in different selections of primary effects. The context for a given 
selection is determined by the initial and goal states used in the training 
process. An interesting research issue is how to compress the different selec- 
tions of primary effects into a single, compact operator representation. One 
idea is to combine primary effects with the conditional effects of an extended 
operator representation, to yield a more effective operator representation. 



13.7 Summary 

In this chapter we have developed a formal understanding of planning with 
primary effects and addressed the issue of how primary effects could be se- 
lected based on a set of training examples. We pointed out two problems 
associated with primary effects, namely the loss of completeness for a plan- 
ner and the possibility of sub-optimality. In addressing the issues, we have 
introduced a PAC model of primary effects, and based on the model, we de- 
scribed an inductive-learning algorithm for automatically choosing them for 
improving the solution quality. 



13.8 Background 

Machine learning and planning have a long history of working together. For a 
comprehensive overview, see Minton’s book [97]. However, few attempts have 
been made to combine PAC learning and planning, as we have done in this 
chapter. 

The material in this chapter is adapted from two IJCAI papers with Eu- 
gene Fink, [47] and [48]. The latter contains extensive experimental results 
confirming the utility of using primary effects in planning. The learning al- 
gorithm and the PAC proof contained in this chapter have been considerably 
modified to simplify the presentation. 

Much of the author’s knowledge in PAC learning has been the result of 
enjoyable discussions with Ming Li. For an introduction to PAC learning 
theory, see books by Anthony and Biggs [8] and by Kearns and Vazirani [78]. 
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